2024-04-24T22:54:46.791 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T22:54:46.791 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178 --owner scheduled_teuthology@teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/orig.config.yaml 2024-04-24T22:54:46.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:54:46.952 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672178' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - SLOW_OPS - slow request sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBydqJySd+F8T4Tf+cO/wcejWdOIiaL3TdjDVNRcZihIEGs2w+P5uAZRpzna4hp+CXTYxGwSrA/e2545QofpySg= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAQOhK0sfGy1K+WIwEYS4iuhhAkKa1lvT/Z1FlSmGJiJrkjO085VsGAtHrF0/jm8AulHcPmwrmon4Fsotj0h5R4= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNDXiVbipmkM6Ls8g762C5D9BpHOeAuIY6huVsbHXcjBjI+yGDoJM9zrIE9Zd4AnOkxD+s8dedRiycIkRQGe5Yg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T22:54:46.952 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T22:54:46.953 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T22:54:46.953 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T22:54:46.955 INFO:teuthology.task.internal:Checking packages... 2024-04-24T22:54:46.987 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T22:54:46.988 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:54:46.988 INFO:teuthology.packaging:ref: None 2024-04-24T22:54:46.988 INFO:teuthology.packaging:tag: None 2024-04-24T22:54:46.988 INFO:teuthology.packaging:branch: squid 2024-04-24T22:54:46.988 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:54:46.988 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T22:54:47.185 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T22:54:47.187 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T22:54:47.194 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T22:54:47.194 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T22:54:47.216 INFO:teuthology.task.internal:Saving configuration 2024-04-24T22:54:47.240 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T22:54:47.246 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T22:54:47.270 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:49:33.190592', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBydqJySd+F8T4Tf+cO/wcejWdOIiaL3TdjDVNRcZihIEGs2w+P5uAZRpzna4hp+CXTYxGwSrA/e2545QofpySg='} 2024-04-24T22:54:47.290 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:49:33.191997', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAQOhK0sfGy1K+WIwEYS4iuhhAkKa1lvT/Z1FlSmGJiJrkjO085VsGAtHrF0/jm8AulHcPmwrmon4Fsotj0h5R4='} 2024-04-24T22:54:47.317 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:49:33.189119', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNDXiVbipmkM6Ls8g762C5D9BpHOeAuIY6huVsbHXcjBjI+yGDoJM9zrIE9Zd4AnOkxD+s8dedRiycIkRQGe5Yg='} 2024-04-24T22:54:47.317 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T22:54:47.322 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T22:54:47.323 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T22:54:47.323 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T22:54:47.323 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T22:54:47.424 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f19fd805160>, signals=[15]) 2024-04-24T22:54:47.425 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T22:54:47.431 INFO:teuthology.task.internal:Opening connections... 2024-04-24T22:54:47.431 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T22:54:47.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:54:47.510 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-04-24T22:54:47.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:54:47.584 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-04-24T22:54:47.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:54:47.657 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T22:54:47.664 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-04-24T22:54:47.681 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-04-24T22:54:47.681 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:54:47.738 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:54:47.739 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:54:47.739 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-04-24T22:54:47.780 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-04-24T22:54:47.799 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-04-24T22:54:47.799 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-04-24T22:54:47.856 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-04-24T22:54:47.856 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-04-24T22:54:47.856 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:54:47.857 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:54:47.857 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-04-24T22:54:47.888 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-04-24T22:54:47.906 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-04-24T22:54:47.907 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-04-24T22:54:47.963 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-04-24T22:54:47.963 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:54:47.964 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:54:47.965 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-04-24T22:54:47.988 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T22:54:48.002 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T22:54:48.008 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T22:54:48.009 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:54:48.011 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:54:48.013 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:54:48.029 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T22:54:48.035 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T22:54:48.035 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:54:48.068 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:54:48.071 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:54:48.323 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T22:54:48.396 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T22:54:48.410 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-24T22:54:48.410 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-24T22:54:48.411 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:54:48.411 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:54:48.411 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:54:48.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:54:48.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:54:48.430 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-04-24T22:54:48.430 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-04-24T22:54:48.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:54:48.431 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-04-24T22:54:48.488 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:54:48.488 INFO:teuthology.task.kernel:Running kernel on smithi102: 5.14.0-437.el9.x86_64 2024-04-24T22:54:48.488 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum install -y kernel 2024-04-24T22:54:48.490 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:54:48.490 INFO:teuthology.orchestra.run.smithi161.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:54:48.490 INFO:teuthology.task.kernel:Running kernel on smithi183: 5.14.0-437.el9.x86_64 2024-04-24T22:54:48.490 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum install -y kernel 2024-04-24T22:54:48.491 INFO:teuthology.task.kernel:Running kernel on smithi161: 5.14.0-437.el9.x86_64 2024-04-24T22:54:48.491 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum install -y kernel 2024-04-24T22:54:50.896 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-24T22:54:51.803 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-04-24T22:54:52.706 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-24T22:54:53.705 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.0 MB 00:03 2024-04-24T22:54:54.687 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-24T22:54:56.748 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-24T22:54:57.380 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 7.3 MB/s | 19 MB 00:02 2024-04-24T22:54:58.166 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-24T22:54:59.363 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-04-24T22:54:59.459 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-24T22:55:01.239 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-24T22:55:01.998 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-04-24T22:55:03.766 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-24T22:55:04.567 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-24T22:55:05.174 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-24T22:55:06.414 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:55:06.414 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:55:06.516 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-24T22:55:06.517 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-04-24T22:55:06.517 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-04-24T22:55:06.636 DEBUG:teuthology.orchestra.run.smithi102:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:55:07.136 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:55:04 PM UTC. 2024-04-24T22:55:07.262 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-24T22:55:07.273 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-04-24T22:55:07.273 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-04-24T22:55:07.273 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-04-24T22:55:07.273 INFO:teuthology.orchestra.run.smithi102.stdout:Reinstalling: 2024-04-24T22:55:07.273 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:55:07.273 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T22:55:07.273 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-04-24T22:55:07.273 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-04-24T22:55:07.274 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T22:55:07.274 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 255 k 2024-04-24T22:55:07.274 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 0 2024-04-24T22:55:07.274 INFO:teuthology.orchestra.run.smithi102.stderr:Operation aborted. 2024-04-24T22:55:07.330 INFO:teuthology.orchestra.run.smithi102.stdout:Is this ok [y/N]: 2024-04-24T22:55:07.331 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum reinstall -y kernel || true 2024-04-24T22:55:07.815 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:55:04 PM UTC. 2024-04-24T22:55:07.931 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout:Reinstalling: 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 255 k 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 0 2024-04-24T22:55:07.932 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-04-24T22:55:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-437.el9.x86_64.rpm 532 kB/s | 255 kB 00:00 2024-04-24T22:55:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:55:08.694 INFO:teuthology.orchestra.run.smithi102.stdout:Total 335 kB/s | 255 kB 00:00 2024-04-24T22:55:08.710 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-04-24T22:55:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-04-24T22:55:08.900 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-04-24T22:55:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-04-24T22:55:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-04-24T22:55:09.069 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-04-24T22:55:09.145 INFO:teuthology.orchestra.run.smithi102.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:55:09.145 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:55:09.848 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:55:09.848 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:55:10.266 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-24T22:55:10.550 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:55:10.550 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T22:55:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:Reinstalled: 2024-04-24T22:55:10.550 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:55:10.550 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T22:55:10.550 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-04-24T22:55:10.723 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:55:10.770 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:55:10.771 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:55:10.771 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:55:10.771 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-24T22:55:10.771 DEBUG:teuthology.orchestra.run.smithi102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T22:55:10.811 INFO:teuthology.orchestra.run.smithi102.stdout:ttyS1 2024-04-24T22:55:10.830 DEBUG:teuthology.parallel:result is None 2024-04-24T22:55:12.122 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:55:12.122 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:55:12.217 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-04-24T22:55:12.218 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-04-24T22:55:12.218 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-04-24T22:55:12.311 DEBUG:teuthology.orchestra.run.smithi183:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:55:12.809 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Apr 2024 10:55:10 PM UTC. 2024-04-24T22:55:12.928 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-04-24T22:55:12.942 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-04-24T22:55:12.943 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-04-24T22:55:12.943 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-04-24T22:55:12.943 INFO:teuthology.orchestra.run.smithi183.stdout:Reinstalling: 2024-04-24T22:55:12.943 INFO:teuthology.orchestra.run.smithi183.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:55:12.943 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T22:55:12.944 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-04-24T22:55:12.944 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-04-24T22:55:12.944 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T22:55:12.944 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 255 k 2024-04-24T22:55:12.944 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 0 2024-04-24T22:55:12.945 INFO:teuthology.orchestra.run.smithi183.stderr:Operation aborted. 2024-04-24T22:55:13.007 INFO:teuthology.orchestra.run.smithi183.stdout:Is this ok [y/N]: 2024-04-24T22:55:13.008 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum reinstall -y kernel || true 2024-04-24T22:55:13.497 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:55:10 PM UTC. 2024-04-24T22:55:13.613 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-04-24T22:55:13.614 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-04-24T22:55:13.614 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-04-24T22:55:13.614 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-04-24T22:55:13.614 INFO:teuthology.orchestra.run.smithi183.stdout:Reinstalling: 2024-04-24T22:55:13.615 INFO:teuthology.orchestra.run.smithi183.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:55:13.615 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T22:55:13.615 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-04-24T22:55:13.615 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-04-24T22:55:13.615 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T22:55:13.615 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 255 k 2024-04-24T22:55:13.616 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 0 2024-04-24T22:55:13.616 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-04-24T22:55:14.326 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-437.el9.x86_64.rpm 392 kB/s | 255 kB 00:00 2024-04-24T22:55:14.326 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:55:14.326 INFO:teuthology.orchestra.run.smithi183.stdout:Total 358 kB/s | 255 kB 00:00 2024-04-24T22:55:14.344 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-04-24T22:55:14.539 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-04-24T22:55:14.540 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-04-24T22:55:14.543 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-04-24T22:55:14.543 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-04-24T22:55:14.678 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-04-24T22:55:14.752 INFO:teuthology.orchestra.run.smithi183.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:55:14.752 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:55:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:55:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:55:16.050 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:55:16.050 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T22:55:16.050 INFO:teuthology.orchestra.run.smithi183.stdout:Reinstalled: 2024-04-24T22:55:16.050 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:55:16.050 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T22:55:16.050 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-04-24T22:55:16.229 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:55:16.276 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:55:16.276 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:55:16.276 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:55:16.276 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-24T22:55:16.277 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T22:55:16.303 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2024-04-24T22:55:16.326 DEBUG:teuthology.parallel:result is None 2024-04-24T22:55:20.549 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-24T22:55:25.662 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-24T22:55:27.560 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:55:27.561 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:55:27.649 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-04-24T22:55:27.650 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-04-24T22:55:27.650 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-04-24T22:55:27.760 DEBUG:teuthology.orchestra.run.smithi161:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:55:28.264 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:55:25 PM UTC. 2024-04-24T22:55:28.388 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-04-24T22:55:28.396 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-04-24T22:55:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-04-24T22:55:28.396 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-04-24T22:55:28.396 INFO:teuthology.orchestra.run.smithi161.stdout:Reinstalling: 2024-04-24T22:55:28.396 INFO:teuthology.orchestra.run.smithi161.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:55:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T22:55:28.397 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-04-24T22:55:28.397 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-04-24T22:55:28.397 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T22:55:28.397 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 255 k 2024-04-24T22:55:28.397 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 0 2024-04-24T22:55:28.397 INFO:teuthology.orchestra.run.smithi161.stderr:Operation aborted. 2024-04-24T22:55:28.463 INFO:teuthology.orchestra.run.smithi161.stdout:Is this ok [y/N]: 2024-04-24T22:55:28.463 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum reinstall -y kernel || true 2024-04-24T22:55:28.961 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:55:25 PM UTC. 2024-04-24T22:55:29.077 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout:Reinstalling: 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 255 k 2024-04-24T22:55:29.078 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 0 2024-04-24T22:55:29.079 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-04-24T22:55:29.656 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-5.14.0-437.el9.x86_64.rpm 495 kB/s | 255 kB 00:00 2024-04-24T22:55:29.657 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:55:29.657 INFO:teuthology.orchestra.run.smithi161.stdout:Total 441 kB/s | 255 kB 00:00 2024-04-24T22:55:29.672 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-04-24T22:55:29.869 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-04-24T22:55:29.870 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-04-24T22:55:29.874 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-04-24T22:55:29.875 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-04-24T22:55:30.011 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-04-24T22:55:30.074 INFO:teuthology.orchestra.run.smithi161.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:55:30.074 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:55:30.602 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:55:30.602 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:55:31.317 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:55:31.317 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T22:55:31.317 INFO:teuthology.orchestra.run.smithi161.stdout:Reinstalled: 2024-04-24T22:55:31.317 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:55:31.317 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T22:55:31.317 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-04-24T22:55:31.498 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:55:31.585 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:55:31.586 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:55:31.586 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:55:31.586 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-24T22:55:31.586 DEBUG:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T22:55:31.620 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2024-04-24T22:55:31.639 DEBUG:teuthology.parallel:result is None 2024-04-24T22:55:31.639 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T22:55:31.647 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T22:55:31.647 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:55:31.650 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:55:31.683 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:55:31.701 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T22:55:31.708 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T22:55:31.715 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T22:55:31.715 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:55:31.718 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:55:31.737 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:55:31.819 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T22:55:31.825 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T22:55:31.825 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:55:31.828 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:55:31.830 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:55:31.858 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:55:31.864 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:55:31.865 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:55:31.867 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:55:31.874 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:55:31.875 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:55:31.877 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T22:55:31.884 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T22:55:31.884 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:55:31.918 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:55:31.921 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:55:31.948 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T22:55:31.957 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T22:55:31.958 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:55:31.985 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:55:31.988 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:55:32.004 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:55:32.078 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:55:32.126 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:55:32.206 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:55:32.247 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T22:55:32.247 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:55:32.311 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:55:32.348 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:55:32.405 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:55:32.487 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:55:32.525 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T22:55:32.525 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:55:32.589 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:55:32.633 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:55:32.694 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:55:32.738 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:55:32.779 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T22:55:32.779 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:55:32.846 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-04-24T22:55:32.848 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-04-24T22:55:32.850 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-04-24T22:55:32.879 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:55:32.884 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:55:32.920 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:55:33.393 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T22:55:33.402 INFO:teuthology.task.internal:Starting timer... 2024-04-24T22:55:33.402 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T22:55:33.412 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T22:55:33.420 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T22:55:33.420 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-04-24T22:55:33.488 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-04-24T22:55:33.490 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-04-24T22:55:33.541 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-04-24T22:55:33.543 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-04-24T22:55:33.592 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-04-24T22:55:33.594 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T22:55:33.594 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-04-24T22:55:33.612 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-04-24T22:55:33.612 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-04-24T22:55:33.632 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-04-24T22:55:33.632 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-04-24T22:55:33.657 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-04-24T22:55:33.657 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-04-24T22:55:33.658 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:55:33.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:55:33.683 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:55:33.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:55:33.707 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:55:33.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:55:33.732 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T22:55:33.732 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:55:33.756 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:55:33.783 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:55:33.809 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T22:55:33.819 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T22:55:33.837 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T22:55:33.839 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi102.front.sepia.ceph.com,smithi161.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T22:59:12.482 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-04-24T22:59:12.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-24T22:59:12.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:59:12.559 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-04-24T22:59:12.633 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-24T22:59:12.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-04-24T22:59:12.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:59:12.709 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-04-24T22:59:12.789 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-04-24T22:59:12.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-24T22:59:12.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:59:12.864 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-04-24T22:59:12.942 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-24T22:59:12.943 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T22:59:12.953 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T22:59:12.954 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:59:12.954 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:59:12.957 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:59:12.957 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:59:12.960 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:59:12.960 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:59:12.984 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:59:12.984 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:59:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:59:12.998 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:59:13.014 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:59:13.020 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-04-24T22:59:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-04-24T22:59:13.028 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:59:13.030 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-04-24T22:59:13.031 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-04-24T22:59:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:59:13.043 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:59:13.052 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-04-24T22:59:13.058 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:59:13.058 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:59:13.063 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-04-24T22:59:13.077 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:59:13.090 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:59:13.110 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:59:13.112 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:59:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:59:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-24T22:59:13.114 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.115 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.117 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:59:13.117 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-04-24T22:59:13.117 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.117 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.117 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.117 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.146 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:59:13.149 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:59:13.149 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-04-24T22:59:13.149 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.149 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.149 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.149 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:59:13.150 INFO:teuthology.run_tasks:Running task install... 2024-04-24T22:59:13.158 DEBUG:teuthology.task.install:project ceph 2024-04-24T22:59:13.158 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T22:59:13.159 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T22:59:13.159 INFO:teuthology.task.install:Using flavor: default 2024-04-24T22:59:13.167 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T22:59:13.167 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T22:59:13.168 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:59:13.168 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:59:13.170 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:59:13.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:59:13.171 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:59:13.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:59:13.339 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:59:13.339 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:59:13.362 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:59:13.362 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:59:13.412 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:59:13.413 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:59:13.509 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:59:13.509 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T22:59:13.509 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:59:13.521 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:59:13.521 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T22:59:13.521 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:59:13.528 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:59:13.528 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T22:59:13.529 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:59:13.534 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:59:13.534 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:59:13.546 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:59:13.546 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:59:13.553 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:59:13.553 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:59:13.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:59:13.612 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:59:13.619 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:59:13.675 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:59:13.687 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:59:13.692 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:59:13.737 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-04-24T22:59:13.739 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-04-24T22:59:13.749 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-04-24T22:59:13.751 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-04-24T22:59:13.754 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-04-24T22:59:13.756 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-04-24T22:59:13.965 INFO:teuthology.orchestra.run.smithi161.stdout:48 files removed 2024-04-24T22:59:13.973 INFO:teuthology.orchestra.run.smithi102.stdout:48 files removed 2024-04-24T22:59:13.979 INFO:teuthology.orchestra.run.smithi183.stdout:48 files removed 2024-04-24T22:59:13.992 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:59:13.999 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:59:14.006 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:59:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-04-24T22:59:14.673 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-04-24T22:59:14.729 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-04-24T22:59:15.032 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-04-24T22:59:15.034 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-04-24T22:59:15.154 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-04-24T22:59:15.386 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-24T22:59:15.391 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-04-24T22:59:15.572 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-24T22:59:16.550 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.0 MB 00:01 2024-04-24T22:59:16.722 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.0 MB 00:01 2024-04-24T22:59:16.952 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-24T22:59:18.498 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-24T22:59:18.978 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-24T22:59:23.021 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-24T22:59:24.388 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-04-24T22:59:24.774 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.2 MB 00:02 2024-04-24T22:59:25.765 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-24T22:59:26.258 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-24T22:59:28.320 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-24T22:59:30.798 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-24T22:59:32.925 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:32.925 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:32.936 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:59:32.937 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:59:32.938 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:59:32.938 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:59:32.940 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:59:32.942 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:59:32.943 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:59:33.002 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-24T22:59:33.011 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:59:33.012 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T22:59:33.013 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T22:59:33.014 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:59:33.015 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:59:33.016 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:59:33.017 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:59:33.017 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:59:33.017 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:59:33.017 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:59:33.017 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:59:33.017 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:59:33.017 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:59:33.017 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:59:33.017 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:59:33.018 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:59:33.020 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:59:33.021 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:59:33.022 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-04-24T22:59:33.023 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:59:33.024 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T22:59:33.024 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-04-24T22:59:33.024 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-24T22:59:33.024 INFO:teuthology.orchestra.run.smithi102.stdout:Install 148 Packages 2024-04-24T22:59:33.024 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 4 Packages 2024-04-24T22:59:33.024 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T22:59:33.024 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 220 M 2024-04-24T22:59:33.024 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-04-24T22:59:33.384 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-24T22:59:33.857 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-04-24T22:59:33.941 INFO:teuthology.orchestra.run.smithi102.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-24T22:59:34.158 INFO:teuthology.orchestra.run.smithi102.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 4.0 MB/s | 890 kB 00:00 2024-04-24T22:59:34.216 INFO:teuthology.orchestra.run.smithi102.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.4 MB/s | 145 kB 00:00 2024-04-24T22:59:34.450 INFO:teuthology.orchestra.run.smithi102.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 7.8 MB/s | 5.4 MB 00:00 2024-04-24T22:59:34.525 INFO:teuthology.orchestra.run.smithi102.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 7.7 MB/s | 2.4 MB 00:00 2024-04-24T22:59:34.617 INFO:teuthology.orchestra.run.smithi102.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-04-24T22:59:35.026 INFO:teuthology.orchestra.run.smithi102.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 9.4 MB/s | 4.7 MB 00:00 2024-04-24T22:59:35.544 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:35.545 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:35.558 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:59:35.558 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:59:35.558 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:59:35.558 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:59:35.560 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:59:35.562 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:59:35.563 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:59:35.625 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-04-24T22:59:35.634 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-04-24T22:59:35.634 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-04-24T22:59:35.634 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-04-24T22:59:35.634 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-04-24T22:59:35.634 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:59:35.634 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:59:35.634 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:59:35.634 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:59:35.635 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:59:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:59:35.637 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:59:35.638 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:59:35.639 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:59:35.640 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:59:35.641 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:59:35.641 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:59:35.641 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:59:35.641 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:59:35.641 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:59:35.641 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:59:35.641 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:59:35.641 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:59:35.641 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:59:35.642 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:59:35.643 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:59:35.644 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:59:35.645 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:59:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-04-24T22:59:35.647 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-04-24T22:59:35.648 INFO:teuthology.orchestra.run.smithi161.stdout:Install 148 Packages 2024-04-24T22:59:35.648 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 4 Packages 2024-04-24T22:59:35.648 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T22:59:35.648 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 220 M 2024-04-24T22:59:35.648 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-04-24T22:59:35.968 INFO:teuthology.orchestra.run.smithi102.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 12 MB/s | 17 MB 00:01 2024-04-24T22:59:36.126 INFO:teuthology.orchestra.run.smithi102.stdout:(9/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 9.3 MB/s | 10 MB 00:01 2024-04-24T22:59:36.160 INFO:teuthology.orchestra.run.smithi102.stdout:(10/152): ceph-selinux-19.0.0-2445.gd2c45d97.el 131 kB/s | 25 kB 00:00 2024-04-24T22:59:36.244 INFO:teuthology.orchestra.run.smithi102.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.6 MB/s | 222 kB 00:00 2024-04-24T22:59:36.302 INFO:teuthology.orchestra.run.smithi102.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 464 kB/s | 27 kB 00:00 2024-04-24T22:59:36.362 INFO:teuthology.orchestra.run.smithi102.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 539 kB/s | 32 kB 00:00 2024-04-24T22:59:36.454 INFO:teuthology.orchestra.run.smithi102.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 8.0 MB/s | 745 kB 00:00 2024-04-24T22:59:36.520 INFO:teuthology.orchestra.run.smithi102.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 163 kB 00:00 2024-04-24T22:59:36.587 INFO:teuthology.orchestra.run.smithi102.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.9 MB/s | 127 kB 00:00 2024-04-24T22:59:36.671 INFO:teuthology.orchestra.run.smithi102.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.9 MB/s | 501 kB 00:00 2024-04-24T22:59:36.710 INFO:teuthology.orchestra.run.smithi161.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-04-24T22:59:37.196 INFO:teuthology.orchestra.run.smithi102.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 10 MB/s | 5.3 MB 00:00 2024-04-24T22:59:37.288 INFO:teuthology.orchestra.run.smithi102.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 493 kB/s | 45 kB 00:00 2024-04-24T22:59:37.372 INFO:teuthology.orchestra.run.smithi102.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.6 MB/s | 133 kB 00:00 2024-04-24T22:59:37.427 INFO:teuthology.orchestra.run.smithi161.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.2 MB/s | 890 kB 00:00 2024-04-24T22:59:37.447 INFO:teuthology.orchestra.run.smithi102.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.1 MB/s | 162 kB 00:00 2024-04-24T22:59:37.531 INFO:teuthology.orchestra.run.smithi102.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.8 MB/s | 321 kB 00:00 2024-04-24T22:59:37.544 INFO:teuthology.orchestra.run.smithi161.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.2 MB/s | 145 kB 00:00 2024-04-24T22:59:37.614 INFO:teuthology.orchestra.run.smithi102.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.5 MB/s | 299 kB 00:00 2024-04-24T22:59:37.706 INFO:teuthology.orchestra.run.smithi102.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.1 MB/s | 99 kB 00:00 2024-04-24T22:59:37.781 INFO:teuthology.orchestra.run.smithi102.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-24T22:59:38.232 INFO:teuthology.orchestra.run.smithi102.stdout:(26/152): ceph-common-19.0.0-2445.gd2c45d97.el9 4.7 MB/s | 21 MB 00:04 2024-04-24T22:59:38.278 INFO:teuthology.orchestra.run.smithi161.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 3.2 MB/s | 2.4 MB 00:00 2024-04-24T22:59:38.290 INFO:teuthology.orchestra.run.smithi102.stdout:(27/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 6.1 MB/s | 3.1 MB 00:00 2024-04-24T22:59:38.315 INFO:teuthology.orchestra.run.smithi102.stdout:(28/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-24T22:59:38.341 INFO:teuthology.orchestra.run.smithi102.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 633 kB/s | 32 kB 00:00 2024-04-24T22:59:38.399 INFO:teuthology.orchestra.run.smithi102.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.7 MB/s | 141 kB 00:00 2024-04-24T22:59:38.603 INFO:teuthology.orchestra.run.smithi161.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.6 MB/s | 1.5 MB 00:00 2024-04-24T22:59:38.767 INFO:teuthology.orchestra.run.smithi102.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 8.3 MB/s | 3.5 MB 00:00 2024-04-24T22:59:38.933 INFO:teuthology.orchestra.run.smithi102.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.5 MB/s | 249 kB 00:00 2024-04-24T22:59:39.017 INFO:teuthology.orchestra.run.smithi102.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 592 kB/s | 49 kB 00:00 2024-04-24T22:59:39.101 INFO:teuthology.orchestra.run.smithi102.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 184 kB/s | 15 kB 00:00 2024-04-24T22:59:39.184 INFO:teuthology.orchestra.run.smithi102.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.1 MB/s | 264 kB 00:00 2024-04-24T22:59:39.186 INFO:teuthology.orchestra.run.smithi161.stdout:(6/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-24T22:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-24T22:59:39.318 INFO:teuthology.orchestra.run.smithi102.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 5.6 MB/s | 762 kB 00:00 2024-04-24T22:59:39.402 INFO:teuthology.orchestra.run.smithi102.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 218 kB/s | 18 kB 00:00 2024-04-24T22:59:39.502 INFO:teuthology.orchestra.run.smithi102.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 805 kB/s | 80 kB 00:00 2024-04-24T22:59:39.611 INFO:teuthology.orchestra.run.smithi102.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-04-24T22:59:39.661 INFO:teuthology.orchestra.run.smithi102.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-24T22:59:39.711 INFO:teuthology.orchestra.run.smithi102.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-24T22:59:39.787 INFO:teuthology.orchestra.run.smithi102.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-24T22:59:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-24T22:59:39.879 INFO:teuthology.orchestra.run.smithi102.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-24T22:59:39.929 INFO:teuthology.orchestra.run.smithi102.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-24T22:59:39.979 INFO:teuthology.orchestra.run.smithi102.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-24T22:59:40.295 INFO:teuthology.orchestra.run.smithi161.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 10 MB/s | 10 MB 00:00 2024-04-24T22:59:40.322 INFO:teuthology.orchestra.run.smithi102.stdout:(47/152): ceph-mgr-diskprediction-local-19.0.0- 3.8 MB/s | 7.4 MB 00:01 2024-04-24T22:59:40.337 INFO:teuthology.orchestra.run.smithi161.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 606 kB/s | 25 kB 00:00 2024-04-24T22:59:40.355 INFO:teuthology.orchestra.run.smithi102.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-24T22:59:40.772 INFO:teuthology.orchestra.run.smithi102.stdout:(49/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 10 MB/s | 48 MB 00:04 2024-04-24T22:59:40.797 INFO:teuthology.orchestra.run.smithi102.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 242 kB/s | 107 kB 00:00 2024-04-24T22:59:40.914 INFO:teuthology.orchestra.run.smithi102.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 152 kB/s | 21 kB 00:00 2024-04-24T22:59:40.940 INFO:teuthology.orchestra.run.smithi102.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-24T22:59:40.998 INFO:teuthology.orchestra.run.smithi102.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-04-24T22:59:41.056 INFO:teuthology.orchestra.run.smithi102.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-24T22:59:41.090 INFO:teuthology.orchestra.run.smithi102.stdout:(55/152): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-04-24T22:59:41.107 INFO:teuthology.orchestra.run.smithi102.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-04-24T22:59:41.166 INFO:teuthology.orchestra.run.smithi102.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-04-24T22:59:41.191 INFO:teuthology.orchestra.run.smithi102.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-24T22:59:41.333 INFO:teuthology.orchestra.run.smithi102.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-04-24T22:59:41.383 INFO:teuthology.orchestra.run.smithi102.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-04-24T22:59:41.434 INFO:teuthology.orchestra.run.smithi102.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-24T22:59:41.499 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.2 MB 00:04 2024-04-24T22:59:41.500 INFO:teuthology.orchestra.run.smithi102.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 1.8 MB/s | 582 kB 00:00 2024-04-24T22:59:41.525 INFO:teuthology.orchestra.run.smithi102.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-24T22:59:41.584 INFO:teuthology.orchestra.run.smithi102.stdout:(64/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-04-24T22:59:41.683 INFO:teuthology.orchestra.run.smithi102.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 225 kB/s | 35 kB 00:00 2024-04-24T22:59:41.712 INFO:teuthology.orchestra.run.smithi102.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 892 kB/s | 188 kB 00:00 2024-04-24T22:59:42.154 INFO:teuthology.orchestra.run.smithi102.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.3 MB/s | 4.7 MB 00:00 2024-04-24T22:59:42.204 INFO:teuthology.orchestra.run.smithi102.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 3.2 MB/s | 1.6 MB 00:00 2024-04-24T22:59:42.296 INFO:teuthology.orchestra.run.smithi102.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-04-24T22:59:42.321 INFO:teuthology.orchestra.run.smithi102.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-24T22:59:42.346 INFO:teuthology.orchestra.run.smithi102.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 911 kB 00:00 2024-04-24T22:59:42.405 INFO:teuthology.orchestra.run.smithi102.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-24T22:59:42.580 INFO:teuthology.orchestra.run.smithi102.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-04-24T22:59:42.672 INFO:teuthology.orchestra.run.smithi102.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-24T22:59:42.773 INFO:teuthology.orchestra.run.smithi102.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-24T22:59:42.845 INFO:teuthology.orchestra.run.smithi161.stdout:(10/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 19 MB/s | 48 MB 00:02 2024-04-24T22:59:42.848 INFO:teuthology.orchestra.run.smithi102.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-04-24T22:59:42.865 INFO:teuthology.orchestra.run.smithi102.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 279 kB/s | 26 kB 00:00 2024-04-24T22:59:42.895 INFO:teuthology.orchestra.run.smithi161.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 4.4 MB/s | 222 kB 00:00 2024-04-24T22:59:42.940 INFO:teuthology.orchestra.run.smithi102.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-24T22:59:42.946 INFO:teuthology.orchestra.run.smithi161.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 542 kB/s | 27 kB 00:00 2024-04-24T22:59:42.957 INFO:teuthology.orchestra.run.smithi102.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-04-24T22:59:42.982 INFO:teuthology.orchestra.run.smithi102.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-04-24T22:59:42.983 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-24T22:59:42.988 INFO:teuthology.orchestra.run.smithi161.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 762 kB/s | 32 kB 00:00 2024-04-24T22:59:42.999 INFO:teuthology.orchestra.run.smithi102.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-04-24T22:59:43.058 INFO:teuthology.orchestra.run.smithi102.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-24T22:59:43.083 INFO:teuthology.orchestra.run.smithi102.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-24T22:59:43.088 INFO:teuthology.orchestra.run.smithi161.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 7.3 MB/s | 745 kB 00:00 2024-04-24T22:59:43.117 INFO:teuthology.orchestra.run.smithi102.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T22:59:43.134 INFO:teuthology.orchestra.run.smithi102.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-04-24T22:59:43.138 INFO:teuthology.orchestra.run.smithi161.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 3.2 MB/s | 163 kB 00:00 2024-04-24T22:59:43.178 INFO:teuthology.orchestra.run.smithi102.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-24T22:59:43.189 INFO:teuthology.orchestra.run.smithi161.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 2.5 MB/s | 127 kB 00:00 2024-04-24T22:59:43.201 INFO:teuthology.orchestra.run.smithi102.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-24T22:59:43.234 INFO:teuthology.orchestra.run.smithi102.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-24T22:59:43.256 INFO:teuthology.orchestra.run.smithi161.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 7.4 MB/s | 501 kB 00:00 2024-04-24T22:59:43.260 INFO:teuthology.orchestra.run.smithi102.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-04-24T22:59:43.293 INFO:teuthology.orchestra.run.smithi102.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-04-24T22:59:43.360 INFO:teuthology.orchestra.run.smithi102.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-24T22:59:43.385 INFO:teuthology.orchestra.run.smithi102.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 454 kB/s | 42 kB 00:00 2024-04-24T22:59:43.419 INFO:teuthology.orchestra.run.smithi102.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-04-24T22:59:43.453 INFO:teuthology.orchestra.run.smithi102.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-24T22:59:43.478 INFO:teuthology.orchestra.run.smithi102.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-04-24T22:59:43.512 INFO:teuthology.orchestra.run.smithi102.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-04-24T22:59:43.537 INFO:teuthology.orchestra.run.smithi102.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-24T22:59:43.564 INFO:teuthology.orchestra.run.smithi161.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 17 MB/s | 5.3 MB 00:00 2024-04-24T22:59:43.571 INFO:teuthology.orchestra.run.smithi102.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-04-24T22:59:43.596 INFO:teuthology.orchestra.run.smithi102.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-24T22:59:43.614 INFO:teuthology.orchestra.run.smithi161.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 910 kB/s | 45 kB 00:00 2024-04-24T22:59:43.630 INFO:teuthology.orchestra.run.smithi102.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T22:59:43.655 INFO:teuthology.orchestra.run.smithi102.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-04-24T22:59:43.682 INFO:teuthology.orchestra.run.smithi161.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.9 MB/s | 133 kB 00:00 2024-04-24T22:59:43.732 INFO:teuthology.orchestra.run.smithi161.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 3.2 MB/s | 162 kB 00:00 2024-04-24T22:59:43.794 INFO:teuthology.orchestra.run.smithi161.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 5.4 MB/s | 321 kB 00:00 2024-04-24T22:59:43.849 INFO:teuthology.orchestra.run.smithi161.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 5.0 MB/s | 299 kB 00:00 2024-04-24T22:59:43.900 INFO:teuthology.orchestra.run.smithi161.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 99 kB 00:00 2024-04-24T22:59:43.947 INFO:teuthology.orchestra.run.smithi102.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-04-24T22:59:43.952 INFO:teuthology.orchestra.run.smithi161.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.6 MB/s | 85 kB 00:00 2024-04-24T22:59:44.136 INFO:teuthology.orchestra.run.smithi161.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 17 MB/s | 3.1 MB 00:00 2024-04-24T22:59:44.303 INFO:teuthology.orchestra.run.smithi161.stdout:(27/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 3.3 MB/s | 17 MB 00:05 2024-04-24T22:59:44.328 INFO:teuthology.orchestra.run.smithi161.stdout:(28/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 893 kB/s | 171 kB 00:00 2024-04-24T22:59:44.353 INFO:teuthology.orchestra.run.smithi161.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 637 kB/s | 32 kB 00:00 2024-04-24T22:59:44.378 INFO:teuthology.orchestra.run.smithi161.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.7 MB/s | 141 kB 00:00 2024-04-24T22:59:44.439 INFO:teuthology.orchestra.run.smithi102.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-04-24T22:59:44.598 INFO:teuthology.orchestra.run.smithi102.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-24T22:59:44.623 INFO:teuthology.orchestra.run.smithi102.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 811 kB/s | 548 kB 00:00 2024-04-24T22:59:44.820 INFO:teuthology.orchestra.run.smithi161.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-04-24T22:59:44.879 INFO:teuthology.orchestra.run.smithi161.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 4.2 MB/s | 249 kB 00:00 2024-04-24T22:59:44.906 INFO:teuthology.orchestra.run.smithi102.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 63 kB/s | 19 kB 00:00 2024-04-24T22:59:44.929 INFO:teuthology.orchestra.run.smithi161.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 989 kB/s | 49 kB 00:00 2024-04-24T22:59:44.980 INFO:teuthology.orchestra.run.smithi161.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 307 kB/s | 15 kB 00:00 2024-04-24T22:59:45.038 INFO:teuthology.orchestra.run.smithi161.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 4.4 MB/s | 264 kB 00:00 2024-04-24T22:59:45.082 INFO:teuthology.orchestra.run.smithi102.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-04-24T22:59:45.107 INFO:teuthology.orchestra.run.smithi102.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-04-24T22:59:45.149 INFO:teuthology.orchestra.run.smithi102.stdout:(109/152): gperftools-libs-2.9.1-2.el9.x86_64.r 435 kB/s | 309 kB 00:00 2024-04-24T22:59:45.174 INFO:teuthology.orchestra.run.smithi102.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 527 kB/s | 49 kB 00:00 2024-04-24T22:59:45.247 INFO:teuthology.orchestra.run.smithi161.stdout:(36/152): ceph-common-19.0.0-2445.gd2c45d97.el9 2.4 MB/s | 21 MB 00:08 2024-04-24T22:59:45.291 INFO:teuthology.orchestra.run.smithi102.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-24T22:59:45.306 INFO:teuthology.orchestra.run.smithi161.stdout:(37/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.7 MB/s | 3.5 MB 00:00 2024-04-24T22:59:45.316 INFO:teuthology.orchestra.run.smithi102.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-04-24T22:59:45.330 INFO:teuthology.orchestra.run.smithi161.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 219 kB/s | 18 kB 00:00 2024-04-24T22:59:45.333 INFO:teuthology.orchestra.run.smithi102.stdout:(113/152): libunwind-1.6.2-1.el9.x86_64.rpm 298 kB/s | 67 kB 00:00 2024-04-24T22:59:45.364 INFO:teuthology.orchestra.run.smithi161.stdout:(39/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 2.3 MB/s | 762 kB 00:00 2024-04-24T22:59:45.425 INFO:teuthology.orchestra.run.smithi102.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-24T22:59:45.468 INFO:teuthology.orchestra.run.smithi102.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 241 kB/s | 32 kB 00:00 2024-04-24T22:59:45.489 INFO:teuthology.orchestra.run.smithi161.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-04-24T22:59:45.518 INFO:teuthology.orchestra.run.smithi102.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-24T22:59:45.543 INFO:teuthology.orchestra.run.smithi102.stdout:(117/152): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-04-24T22:59:45.556 INFO:teuthology.orchestra.run.smithi161.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-24T22:59:45.598 INFO:teuthology.orchestra.run.smithi161.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-24T22:59:45.617 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 21 MB 00:02 2024-04-24T22:59:45.626 INFO:teuthology.orchestra.run.smithi102.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-24T22:59:45.632 INFO:teuthology.orchestra.run.smithi161.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-24T22:59:45.657 INFO:teuthology.orchestra.run.smithi161.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-24T22:59:45.660 INFO:teuthology.orchestra.run.smithi102.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-24T22:59:45.693 INFO:teuthology.orchestra.run.smithi102.stdout:(120/152): python3-cheroot-8.6.0-4.el9.noarch.r 760 kB/s | 172 kB 00:00 2024-04-24T22:59:45.699 INFO:teuthology.orchestra.run.smithi161.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-04-24T22:59:45.724 INFO:teuthology.orchestra.run.smithi161.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-24T22:59:45.727 INFO:teuthology.orchestra.run.smithi102.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-04-24T22:59:45.749 INFO:teuthology.orchestra.run.smithi161.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-24T22:59:45.775 INFO:teuthology.orchestra.run.smithi161.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-24T22:59:45.777 INFO:teuthology.orchestra.run.smithi102.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-24T22:59:45.836 INFO:teuthology.orchestra.run.smithi102.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-24T22:59:45.861 INFO:teuthology.orchestra.run.smithi102.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2024-04-24T22:59:45.920 INFO:teuthology.orchestra.run.smithi102.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-04-24T22:59:45.962 INFO:teuthology.orchestra.run.smithi102.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-24T22:59:46.004 INFO:teuthology.orchestra.run.smithi102.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-24T22:59:46.088 INFO:teuthology.orchestra.run.smithi102.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.2 MB/s | 2.0 MB 00:00 2024-04-24T22:59:46.104 INFO:teuthology.orchestra.run.smithi102.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 138 kB/s | 20 kB 00:00 2024-04-24T22:59:46.147 INFO:teuthology.orchestra.run.smithi102.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-04-24T22:59:46.180 INFO:teuthology.orchestra.run.smithi102.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-04-24T22:59:46.197 INFO:teuthology.orchestra.run.smithi102.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 855 kB/s | 79 kB 00:00 2024-04-24T22:59:46.200 INFO:teuthology.orchestra.run.smithi161.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 251 kB/s | 107 kB 00:00 2024-04-24T22:59:46.256 INFO:teuthology.orchestra.run.smithi102.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-04-24T22:59:46.281 INFO:teuthology.orchestra.run.smithi102.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-24T22:59:46.306 INFO:teuthology.orchestra.run.smithi102.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-24T22:59:46.308 INFO:teuthology.orchestra.run.smithi161.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-04-24T22:59:46.348 INFO:teuthology.orchestra.run.smithi102.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-04-24T22:59:46.374 INFO:teuthology.orchestra.run.smithi102.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-24T22:59:46.399 INFO:teuthology.orchestra.run.smithi102.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-04-24T22:59:46.441 INFO:teuthology.orchestra.run.smithi102.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-24T22:59:46.483 INFO:teuthology.orchestra.run.smithi102.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-04-24T22:59:46.500 INFO:teuthology.orchestra.run.smithi161.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 888 kB/s | 689 kB 00:00 2024-04-24T22:59:46.508 INFO:teuthology.orchestra.run.smithi102.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-04-24T22:59:46.534 INFO:teuthology.orchestra.run.smithi161.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-04-24T22:59:46.550 INFO:teuthology.orchestra.run.smithi102.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-04-24T22:59:46.568 INFO:teuthology.orchestra.run.smithi161.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-04-24T22:59:46.584 INFO:teuthology.orchestra.run.smithi102.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-24T22:59:46.610 INFO:teuthology.orchestra.run.smithi161.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 299 kB/s | 33 kB 00:00 2024-04-24T22:59:46.634 INFO:teuthology.orchestra.run.smithi102.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 714 kB/s | 90 kB 00:00 2024-04-24T22:59:46.659 INFO:teuthology.orchestra.run.smithi102.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-24T22:59:46.669 INFO:teuthology.orchestra.run.smithi161.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-04-24T22:59:46.676 INFO:teuthology.orchestra.run.smithi102.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-24T22:59:46.710 INFO:teuthology.orchestra.run.smithi161.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-04-24T22:59:46.769 INFO:teuthology.orchestra.run.smithi161.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-24T22:59:46.785 INFO:teuthology.orchestra.run.smithi102.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-24T22:59:46.811 INFO:teuthology.orchestra.run.smithi161.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-24T22:59:46.835 INFO:teuthology.orchestra.run.smithi102.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-04-24T22:59:47.020 INFO:teuthology.orchestra.run.smithi161.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.2 MB/s | 3.0 MB 00:00 2024-04-24T22:59:47.045 INFO:teuthology.orchestra.run.smithi161.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.4 MB/s | 582 kB 00:00 2024-04-24T22:59:47.070 INFO:teuthology.orchestra.run.smithi161.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-04-24T22:59:47.077 INFO:teuthology.orchestra.run.smithi102.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 1.3 MB 00:00 2024-04-24T22:59:47.112 INFO:teuthology.orchestra.run.smithi161.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-04-24T22:59:47.145 INFO:teuthology.orchestra.run.smithi161.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-04-24T22:59:47.177 INFO:teuthology.orchestra.run.smithi102.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-24T22:59:47.179 INFO:teuthology.orchestra.run.smithi161.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-24T22:59:47.238 INFO:teuthology.orchestra.run.smithi161.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 383 kB/s | 35 kB 00:00 2024-04-24T22:59:47.380 INFO:teuthology.orchestra.run.smithi161.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 704 kB/s | 188 kB 00:00 2024-04-24T22:59:47.513 INFO:teuthology.orchestra.run.smithi161.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-24T22:59:48.169 INFO:teuthology.orchestra.run.smithi102.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 2.3 MB/s | 3.4 MB 00:01 2024-04-24T22:59:48.605 INFO:teuthology.orchestra.run.smithi161.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.3 MB/s | 4.7 MB 00:01 2024-04-24T22:59:48.752 INFO:teuthology.orchestra.run.smithi102.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 1.6 MB/s | 3.1 MB 00:01 2024-04-24T22:59:48.755 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:59:48.755 INFO:teuthology.orchestra.run.smithi102.stdout:Total 14 MB/s | 220 MB 00:15 2024-04-24T22:59:48.781 INFO:teuthology.orchestra.run.smithi161.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 719 kB/s | 911 kB 00:01 2024-04-24T22:59:48.889 INFO:teuthology.orchestra.run.smithi161.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 470 kB/s | 133 kB 00:00 2024-04-24T22:59:49.849 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-04-24T22:59:49.991 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-04-24T22:59:49.991 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-04-24T22:59:50.148 INFO:teuthology.orchestra.run.smithi161.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 4.4 MB 00:02 2024-04-24T22:59:50.256 INFO:teuthology.orchestra.run.smithi161.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 756 kB/s | 1.0 MB 00:01 2024-04-24T22:59:50.332 INFO:teuthology.orchestra.run.smithi161.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-04-24T22:59:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 715 kB/s | 244 kB 00:00 2024-04-24T22:59:50.726 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-24T22:59:51.015 INFO:teuthology.orchestra.run.smithi161.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 725 kB/s | 248 kB 00:00 2024-04-24T22:59:51.115 INFO:teuthology.orchestra.run.smithi161.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-04-24T22:59:51.216 INFO:teuthology.orchestra.run.smithi161.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-04-24T22:59:51.491 INFO:teuthology.orchestra.run.smithi161.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 626 kB/s | 172 kB 00:00 2024-04-24T22:59:51.608 INFO:teuthology.orchestra.run.smithi161.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 298 kB/s | 35 kB 00:00 2024-04-24T22:59:51.752 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-04-24T22:59:51.752 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-04-24T22:59:52.820 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:52.820 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:59:52.831 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:59:52.832 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:59:52.832 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:59:52.833 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:59:52.835 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:59:52.837 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:59:52.838 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:59:52.901 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-04-24T22:59:52.910 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-24T22:59:52.910 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-04-24T22:59:52.910 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-24T22:59:52.910 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-04-24T22:59:52.910 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:59:52.910 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:59:52.910 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:59:52.910 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T22:59:52.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:59:52.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:59:52.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:59:52.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:59:52.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:59:52.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:59:52.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:59:52.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:59:52.912 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-04-24T22:59:52.913 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:59:52.914 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:59:52.915 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:59:52.916 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:59:52.917 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:59:52.918 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:59:52.919 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:59:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:59:52.921 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 4 Packages 2024-04-24T22:59:52.922 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T22:59:52.923 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 220 M 2024-04-24T22:59:52.923 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-04-24T22:59:53.729 INFO:teuthology.orchestra.run.smithi183.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-24T22:59:54.579 INFO:teuthology.orchestra.run.smithi183.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 890 kB 00:00 2024-04-24T22:59:54.754 INFO:teuthology.orchestra.run.smithi183.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 828 kB/s | 145 kB 00:00 2024-04-24T22:59:55.855 INFO:teuthology.orchestra.run.smithi183.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-04-24T22:59:56.396 INFO:teuthology.orchestra.run.smithi183.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-04-24T22:59:56.633 INFO:teuthology.orchestra.run.smithi161.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 754 kB/s | 5.8 MB 00:07 2024-04-24T22:59:57.024 INFO:teuthology.orchestra.run.smithi161.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 640 kB/s | 250 kB 00:00 2024-04-24T22:59:57.246 INFO:teuthology.orchestra.run.smithi183.stdout:(6/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-04-24T22:59:57.458 INFO:teuthology.orchestra.run.smithi161.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 513 kB/s | 222 kB 00:00 2024-04-24T22:59:57.463 INFO:teuthology.orchestra.run.smithi183.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 4.4 MB/s | 4.7 MB 00:01 2024-04-24T22:59:57.566 INFO:teuthology.orchestra.run.smithi161.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 712 kB/s | 77 kB 00:00 2024-04-24T22:59:57.717 INFO:teuthology.orchestra.run.smithi161.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 854 kB/s | 5.1 MB 00:06 2024-04-24T22:59:57.735 INFO:teuthology.orchestra.run.smithi161.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 253 kB/s | 42 kB 00:00 2024-04-24T22:59:57.754 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-04-24T22:59:57.826 INFO:teuthology.orchestra.run.smithi161.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-04-24T22:59:57.848 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T22:59:57.935 INFO:teuthology.orchestra.run.smithi161.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-24T22:59:57.970 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T22:59:58.010 INFO:teuthology.orchestra.run.smithi161.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 576 kB/s | 159 kB 00:00 2024-04-24T22:59:58.043 INFO:teuthology.orchestra.run.smithi161.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-04-24T22:59:58.118 INFO:teuthology.orchestra.run.smithi161.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-04-24T22:59:58.167 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T22:59:58.208 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T22:59:58.219 INFO:teuthology.orchestra.run.smithi161.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-04-24T22:59:58.233 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T22:59:58.276 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T22:59:58.328 INFO:teuthology.orchestra.run.smithi161.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-04-24T22:59:58.453 INFO:teuthology.orchestra.run.smithi161.stdout:(93/152): python3-babel-2.9.1-2.el9.noarch.rpm 745 kB/s | 6.0 MB 00:08 2024-04-24T22:59:58.588 INFO:teuthology.orchestra.run.smithi183.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 9.1 MB/s | 10 MB 00:01 2024-04-24T22:59:58.595 INFO:teuthology.orchestra.run.smithi161.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 253 kB/s | 36 kB 00:00 2024-04-24T22:59:58.606 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T22:59:58.672 INFO:teuthology.orchestra.run.smithi183.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 303 kB/s | 25 kB 00:00 2024-04-24T22:59:58.726 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:59:58.729 INFO:teuthology.orchestra.run.smithi161.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 356 kB/s | 47 kB 00:00 2024-04-24T22:59:58.831 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:59:58.868 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:59:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 367 kB/s | 221 kB 00:00 2024-04-24T22:59:58.968 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:59:59.006 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:59:59.054 INFO:teuthology.orchestra.run.smithi161.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 462 kB/s | 58 kB 00:00 2024-04-24T22:59:59.151 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T22:59:59.179 INFO:teuthology.orchestra.run.smithi161.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 751 kB/s | 94 kB 00:00 2024-04-24T22:59:59.203 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T22:59:59.231 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:59:59.271 INFO:teuthology.orchestra.run.smithi161.stdout:(99/152): socat-1.7.4.1-5.el9.x86_64.rpm 562 kB/s | 305 kB 00:00 2024-04-24T22:59:59.334 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:59:59.357 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:59:59.380 INFO:teuthology.orchestra.run.smithi161.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-04-24T22:59:59.405 INFO:teuthology.orchestra.run.smithi161.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 282 kB/s | 64 kB 00:00 2024-04-24T22:59:59.419 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:59:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:59:59.462 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:59:59.497 INFO:teuthology.orchestra.run.smithi161.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-04-24T22:59:59.633 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:59:59.666 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:59:59.697 INFO:teuthology.orchestra.run.smithi161.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-04-24T22:59:59.773 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T22:59:59.798 INFO:teuthology.orchestra.run.smithi161.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-24T22:59:59.856 INFO:teuthology.orchestra.run.smithi161.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-04-24T22:59:59.868 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T22:59:59.881 INFO:teuthology.orchestra.run.smithi161.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-04-24T22:59:59.907 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T22:59:59.923 INFO:teuthology.orchestra.run.smithi161.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 626 kB/s | 26 kB 00:00 2024-04-24T22:59:59.941 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T22:59:59.982 INFO:teuthology.orchestra.run.smithi161.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-04-24T23:00:00.005 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:00:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:00:00.085 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:00:00.116 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:00:00.141 INFO:teuthology.orchestra.run.smithi161.stdout:(109/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.7 MB/s | 16 MB 00:02 2024-04-24T23:00:00.144 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:00:00.166 INFO:teuthology.orchestra.run.smithi161.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-04-24T23:00:00.211 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:00:00.258 INFO:teuthology.orchestra.run.smithi161.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.0 MB/s | 838 kB 00:00 2024-04-24T23:00:00.266 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:00:00.283 INFO:teuthology.orchestra.run.smithi161.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-24T23:00:00.297 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:00:00.333 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:00:00.375 INFO:teuthology.orchestra.run.smithi161.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-24T23:00:00.400 INFO:teuthology.orchestra.run.smithi161.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-04-24T23:00:00.416 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:00:00.426 INFO:teuthology.orchestra.run.smithi161.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-04-24T23:00:00.451 INFO:teuthology.orchestra.run.smithi161.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-24T23:00:00.452 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:00:00.493 INFO:teuthology.orchestra.run.smithi161.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-24T23:00:00.495 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:00:00.524 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:00:00.568 INFO:teuthology.orchestra.run.smithi161.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-24T23:00:00.594 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:00:00.638 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:00:00.677 INFO:teuthology.orchestra.run.smithi161.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-04-24T23:00:00.702 INFO:teuthology.orchestra.run.smithi161.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-24T23:00:00.712 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:00:00.736 INFO:teuthology.orchestra.run.smithi161.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-24T23:00:00.786 INFO:teuthology.orchestra.run.smithi161.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-24T23:00:00.843 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:00:00.844 INFO:teuthology.orchestra.run.smithi161.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-04-24T23:00:00.898 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:00:00.912 INFO:teuthology.orchestra.run.smithi161.stdout:(124/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-04-24T23:00:00.929 INFO:teuthology.orchestra.run.smithi161.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-04-24T23:00:00.963 INFO:teuthology.orchestra.run.smithi161.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-04-24T23:00:00.988 INFO:teuthology.orchestra.run.smithi161.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-24T23:00:01.021 INFO:teuthology.orchestra.run.smithi161.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-04-24T23:00:01.071 INFO:teuthology.orchestra.run.smithi161.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 926 kB/s | 46 kB 00:00 2024-04-24T23:00:01.105 INFO:teuthology.orchestra.run.smithi161.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-24T23:00:01.122 INFO:teuthology.orchestra.run.smithi161.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-24T23:00:01.156 INFO:teuthology.orchestra.run.smithi161.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-24T23:00:01.181 INFO:teuthology.orchestra.run.smithi161.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-24T23:00:01.206 INFO:teuthology.orchestra.run.smithi161.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-24T23:00:01.248 INFO:teuthology.orchestra.run.smithi161.stdout:(135/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-04-24T23:00:01.274 INFO:teuthology.orchestra.run.smithi161.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 975 kB/s | 90 kB 00:00 2024-04-24T23:00:01.299 INFO:teuthology.orchestra.run.smithi161.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-24T23:00:01.324 INFO:teuthology.orchestra.run.smithi161.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-24T23:00:01.349 INFO:teuthology.orchestra.run.smithi161.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-24T23:00:01.375 INFO:teuthology.orchestra.run.smithi161.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-24T23:00:01.399 INFO:teuthology.orchestra.run.smithi161.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-04-24T23:00:01.410 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:00:01.434 INFO:teuthology.orchestra.run.smithi161.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 607 kB/s | 51 kB 00:00 2024-04-24T23:00:01.459 INFO:teuthology.orchestra.run.smithi161.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-24T23:00:01.484 INFO:teuthology.orchestra.run.smithi161.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-24T23:00:01.495 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:00:01.510 INFO:teuthology.orchestra.run.smithi161.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-04-24T23:00:01.524 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:00:01.534 INFO:teuthology.orchestra.run.smithi161.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-24T23:00:01.555 INFO:teuthology.orchestra.run.smithi183.stdout:(10/152): ceph-common-19.0.0-2445.gd2c45d97.el9 2.6 MB/s | 21 MB 00:08 2024-04-24T23:00:01.567 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:00:01.568 INFO:teuthology.orchestra.run.smithi161.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-24T23:00:01.605 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:00:01.627 INFO:teuthology.orchestra.run.smithi161.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-24T23:00:01.646 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:00:01.747 INFO:teuthology.orchestra.run.smithi183.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.1 MB/s | 222 kB 00:00 2024-04-24T23:00:01.792 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:00:01.841 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:00:01.847 INFO:teuthology.orchestra.run.smithi183.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 271 kB/s | 27 kB 00:00 2024-04-24T23:00:01.896 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:00:01.959 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:00:01.964 INFO:teuthology.orchestra.run.smithi183.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 271 kB/s | 32 kB 00:00 2024-04-24T23:00:01.992 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:00:02.029 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:00:02.061 INFO:teuthology.orchestra.run.smithi161.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-04-24T23:00:02.075 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:00:02.111 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:00:02.154 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:00:02.196 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:00:02.281 INFO:teuthology.orchestra.run.smithi183.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 2.3 MB/s | 745 kB 00:00 2024-04-24T23:00:02.330 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:00:02.352 INFO:teuthology.orchestra.run.smithi161.stdout:(150/152): librados2-19.0.0-2445.gd2c45d97.el9. 4.1 MB/s | 3.4 MB 00:00 2024-04-24T23:00:02.377 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:00:02.389 INFO:teuthology.orchestra.run.smithi183.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.5 MB/s | 163 kB 00:00 2024-04-24T23:00:02.407 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:00:02.444 INFO:teuthology.orchestra.run.smithi161.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 2.4 MB 00:00 2024-04-24T23:00:02.448 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:00:02.481 INFO:teuthology.orchestra.run.smithi183.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.4 MB/s | 127 kB 00:00 2024-04-24T23:00:02.559 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:00:02.591 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:00:02.631 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:00:02.656 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:00:02.669 INFO:teuthology.orchestra.run.smithi161.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-24T23:00:02.672 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:00:02.673 INFO:teuthology.orchestra.run.smithi161.stdout:Total 8.1 MB/s | 220 MB 00:27 2024-04-24T23:00:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:00:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:00:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:00:02.890 INFO:teuthology.orchestra.run.smithi183.stdout:(17/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 11 MB/s | 48 MB 00:04 2024-04-24T23:00:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:00:02.924 INFO:teuthology.orchestra.run.smithi183.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 1.1 MB/s | 501 kB 00:00 2024-04-24T23:00:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:00:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:00:02.999 INFO:teuthology.orchestra.run.smithi183.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 603 kB/s | 45 kB 00:00 2024-04-24T23:00:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:00:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:00:03.082 INFO:teuthology.orchestra.run.smithi183.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.6 MB/s | 133 kB 00:00 2024-04-24T23:00:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:00:03.141 INFO:teuthology.orchestra.run.smithi183.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.7 MB/s | 162 kB 00:00 2024-04-24T23:00:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:00:03.275 INFO:teuthology.orchestra.run.smithi183.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 321 kB 00:00 2024-04-24T23:00:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:00:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:00:03.400 INFO:teuthology.orchestra.run.smithi183.stdout:(23/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 10 MB/s | 5.3 MB 00:00 2024-04-24T23:00:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:00:03.425 INFO:teuthology.orchestra.run.smithi183.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 299 kB 00:00 2024-04-24T23:00:03.463 INFO:teuthology.orchestra.run.smithi183.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.7 MB/s | 99 kB 00:00 2024-04-24T23:00:03.485 INFO:teuthology.orchestra.run.smithi183.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-24T23:00:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:00:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:00:03.560 INFO:teuthology.orchestra.run.smithi183.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-24T23:00:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:00:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:00:03.636 INFO:teuthology.orchestra.run.smithi183.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 424 kB/s | 32 kB 00:00 2024-04-24T23:00:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:00:03.736 INFO:teuthology.orchestra.run.smithi183.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.4 MB/s | 141 kB 00:00 2024-04-24T23:00:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:00:03.782 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-04-24T23:00:03.795 INFO:teuthology.orchestra.run.smithi183.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 9.2 MB/s | 3.1 MB 00:00 2024-04-24T23:00:03.928 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-04-24T23:00:03.928 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-04-24T23:00:04.486 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:00:04.531 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:00:04.553 INFO:teuthology.orchestra.run.smithi183.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 9.7 MB/s | 7.4 MB 00:00 2024-04-24T23:00:04.564 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:00:04.620 INFO:teuthology.orchestra.run.smithi183.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.7 MB/s | 249 kB 00:00 2024-04-24T23:00:04.661 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:00:04.695 INFO:teuthology.orchestra.run.smithi183.stdout:(33/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.7 MB/s | 3.5 MB 00:00 2024-04-24T23:00:04.712 INFO:teuthology.orchestra.run.smithi183.stdout:(34/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 536 kB/s | 49 kB 00:00 2024-04-24T23:00:04.713 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:00:04.800 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:00:04.841 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:00:04.871 INFO:teuthology.orchestra.run.smithi183.stdout:(35/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 2.2 MB/s | 17 MB 00:07 2024-04-24T23:00:04.890 INFO:teuthology.orchestra.run.smithi183.stdout:(36/152): ceph-prometheus-alerts-19.0.0-2445.gd 80 kB/s | 15 kB 00:00 2024-04-24T23:00:04.921 INFO:teuthology.orchestra.run.smithi183.stdout:(37/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 1.2 MB/s | 264 kB 00:00 2024-04-24T23:00:04.947 INFO:teuthology.orchestra.run.smithi183.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 312 kB/s | 18 kB 00:00 2024-04-24T23:00:05.010 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:00:05.031 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:00:05.047 INFO:teuthology.orchestra.run.smithi183.stdout:(39/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.2 MB/s | 762 kB 00:00 2024-04-24T23:00:05.108 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:00:05.155 INFO:teuthology.orchestra.run.smithi183.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 344 kB/s | 80 kB 00:00 2024-04-24T23:00:05.322 INFO:teuthology.orchestra.run.smithi183.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-24T23:00:05.348 INFO:teuthology.orchestra.run.smithi183.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-04-24T23:00:05.423 INFO:teuthology.orchestra.run.smithi183.stdout:(43/152): libquadmath-11.4.1-3.el9.x86_64.rpm 520 kB/s | 195 kB 00:00 2024-04-24T23:00:05.448 INFO:teuthology.orchestra.run.smithi183.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-24T23:00:05.507 INFO:teuthology.orchestra.run.smithi183.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-04-24T23:00:05.523 INFO:teuthology.orchestra.run.smithi183.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-24T23:00:05.574 INFO:teuthology.orchestra.run.smithi183.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-24T23:00:05.599 INFO:teuthology.orchestra.run.smithi183.stdout:(48/152): python3-requests-2.25.1-8.el9.noarch. 828 kB/s | 125 kB 00:00 2024-04-24T23:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-04-24T23:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-04-24T23:00:05.742 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:00:05.853 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:00:06.024 INFO:teuthology.orchestra.run.smithi183.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 251 kB/s | 107 kB 00:00 2024-04-24T23:00:06.133 INFO:teuthology.orchestra.run.smithi183.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-04-24T23:00:06.283 INFO:teuthology.orchestra.run.smithi183.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 907 kB/s | 689 kB 00:00 2024-04-24T23:00:06.400 INFO:teuthology.orchestra.run.smithi183.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-04-24T23:00:06.425 INFO:teuthology.orchestra.run.smithi183.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-04-24T23:00:06.451 INFO:teuthology.orchestra.run.smithi183.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 988 kB/s | 313 kB 00:00 2024-04-24T23:00:06.526 INFO:teuthology.orchestra.run.smithi183.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-04-24T23:00:06.560 INFO:teuthology.orchestra.run.smithi183.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-04-24T23:00:06.635 INFO:teuthology.orchestra.run.smithi183.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-24T23:00:06.668 INFO:teuthology.orchestra.run.smithi183.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-04-24T23:00:06.827 INFO:teuthology.orchestra.run.smithi183.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-04-24T23:00:06.952 INFO:teuthology.orchestra.run.smithi183.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.4 MB/s | 3.0 MB 00:00 2024-04-24T23:00:06.978 INFO:teuthology.orchestra.run.smithi183.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 169 kB/s | 25 kB 00:00 2024-04-24T23:00:07.003 INFO:teuthology.orchestra.run.smithi183.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 1.7 MB/s | 582 kB 00:00 2024-04-24T23:00:07.061 INFO:teuthology.orchestra.run.smithi183.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-04-24T23:00:07.079 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:00:07.087 INFO:teuthology.orchestra.run.smithi183.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-24T23:00:07.112 INFO:teuthology.orchestra.run.smithi183.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-24T23:00:07.161 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:00:07.170 INFO:teuthology.orchestra.run.smithi183.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-04-24T23:00:07.188 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:00:07.222 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:00:07.533 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:00:07.576 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:07.667 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:07.698 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:00:07.734 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:00:08.021 INFO:teuthology.orchestra.run.smithi183.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2024-04-24T23:00:08.205 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:00:08.252 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:08.563 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:08.623 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:00:09.304 INFO:teuthology.orchestra.run.smithi183.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 710 kB/s | 911 kB 00:01 2024-04-24T23:00:09.371 INFO:teuthology.orchestra.run.smithi183.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 739 kB/s | 1.6 MB 00:02 2024-04-24T23:00:09.588 INFO:teuthology.orchestra.run.smithi183.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 470 kB/s | 133 kB 00:00 2024-04-24T23:00:09.863 INFO:teuthology.orchestra.run.smithi183.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.7 MB/s | 4.7 MB 00:02 2024-04-24T23:00:10.046 INFO:teuthology.orchestra.run.smithi183.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-04-24T23:00:10.879 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:10.919 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:11.055 INFO:teuthology.orchestra.run.smithi183.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 704 kB/s | 1.0 MB 00:01 2024-04-24T23:00:11.439 INFO:teuthology.orchestra.run.smithi183.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 637 kB/s | 244 kB 00:00 2024-04-24T23:00:11.457 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:11.506 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:11.580 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:00:11.580 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:00:11.580 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:11.607 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:11.619 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-04-24T23:00:11.707 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:00:11.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:00:11.780 INFO:teuthology.orchestra.run.smithi183.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 726 kB/s | 248 kB 00:00 2024-04-24T23:00:11.800 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:00:11.841 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:00:11.867 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:00:11.889 INFO:teuthology.orchestra.run.smithi183.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-04-24T23:00:11.910 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:00:11.989 INFO:teuthology.orchestra.run.smithi183.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-04-24T23:00:12.248 INFO:teuthology.orchestra.run.smithi183.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 667 kB/s | 172 kB 00:00 2024-04-24T23:00:12.267 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:00:12.356 INFO:teuthology.orchestra.run.smithi183.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 323 kB/s | 35 kB 00:00 2024-04-24T23:00:12.384 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:12.507 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:12.535 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:12.627 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:12.656 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:00:12.807 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:00:12.861 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:00:12.881 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:12.985 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:12.999 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:13.062 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:13.075 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:13.095 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:13.324 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:13.431 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:00:13.527 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:00:13.556 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:00:13.591 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:00:13.654 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:00:13.698 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:00:13.742 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:00:13.775 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:00:13.803 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:00:13.870 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:00:13.924 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:00:13.956 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:00:13.991 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:00:14.066 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:00:14.102 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:00:14.145 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:00:14.174 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:00:14.244 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:00:14.297 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:00:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:00:14.502 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:00:14.557 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:00:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:00:15.154 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:00:15.183 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:00:15.225 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:00:15.263 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:00:15.306 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:00:15.470 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:00:15.516 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:00:15.572 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:00:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:00:15.684 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:00:15.720 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:00:15.767 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:00:15.803 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:00:15.846 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:00:15.888 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:00:16.025 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:00:16.060 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:00:16.091 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:00:16.132 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:00:16.226 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:00:16.274 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:00:16.314 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:00:16.340 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:00:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:00:16.439 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:00:16.566 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:00:16.597 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:00:16.625 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:00:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:00:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:00:16.738 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:00:16.768 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:00:16.818 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:00:16.852 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:00:16.894 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:00:16.928 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:00:16.989 INFO:teuthology.orchestra.run.smithi183.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 777 kB/s | 5.8 MB 00:07 2024-04-24T23:00:17.016 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:00:17.059 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:00:17.099 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:00:17.146 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:00:17.245 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:00:17.300 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:00:17.340 INFO:teuthology.orchestra.run.smithi183.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 716 kB/s | 250 kB 00:00 2024-04-24T23:00:17.682 INFO:teuthology.orchestra.run.smithi183.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-04-24T23:00:17.865 INFO:teuthology.orchestra.run.smithi183.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 421 kB/s | 77 kB 00:00 2024-04-24T23:00:17.974 INFO:teuthology.orchestra.run.smithi183.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-04-24T23:00:18.019 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:00:18.063 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:00:18.089 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:00:18.174 INFO:teuthology.orchestra.run.smithi183.stdout:(85/152): python3-babel-2.9.1-2.el9.noarch.rpm 751 kB/s | 6.0 MB 00:08 2024-04-24T23:00:18.186 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:00:18.246 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:00:18.325 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:00:18.358 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:00:18.458 INFO:teuthology.orchestra.run.smithi183.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 556 kB/s | 269 kB 00:00 2024-04-24T23:00:18.483 INFO:teuthology.orchestra.run.smithi183.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 514 kB/s | 159 kB 00:00 2024-04-24T23:00:18.535 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:00:18.564 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:00:18.583 INFO:teuthology.orchestra.run.smithi183.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-04-24T23:00:18.642 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:00:18.683 INFO:teuthology.orchestra.run.smithi183.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-04-24T23:00:18.884 INFO:teuthology.orchestra.run.smithi183.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 655 kB/s | 279 kB 00:00 2024-04-24T23:00:18.992 INFO:teuthology.orchestra.run.smithi183.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-04-24T23:00:19.101 INFO:teuthology.orchestra.run.smithi183.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-04-24T23:00:19.218 INFO:teuthology.orchestra.run.smithi183.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 761 kB/s | 5.1 MB 00:06 2024-04-24T23:00:19.283 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:00:19.326 INFO:teuthology.orchestra.run.smithi183.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-04-24T23:00:19.395 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:00:19.435 INFO:teuthology.orchestra.run.smithi183.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-04-24T23:00:19.486 INFO:teuthology.orchestra.run.smithi183.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 575 kB/s | 221 kB 00:00 2024-04-24T23:00:19.594 INFO:teuthology.orchestra.run.smithi183.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-04-24T23:00:19.786 INFO:teuthology.orchestra.run.smithi183.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 489 kB/s | 94 kB 00:00 2024-04-24T23:00:19.894 INFO:teuthology.orchestra.run.smithi183.stdout:(99/152): socat-1.7.4.1-5.el9.x86_64.rpm 664 kB/s | 305 kB 00:00 2024-04-24T23:00:20.003 INFO:teuthology.orchestra.run.smithi183.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 294 kB/s | 64 kB 00:00 2024-04-24T23:00:20.028 INFO:teuthology.orchestra.run.smithi183.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 335 kB/s | 45 kB 00:00 2024-04-24T23:00:20.103 INFO:teuthology.orchestra.run.smithi183.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-04-24T23:00:20.204 INFO:teuthology.orchestra.run.smithi183.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-24T23:00:20.613 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:00:20.685 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:00:20.712 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:00:20.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:00:20.746 INFO:teuthology.orchestra.run.smithi183.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 854 kB/s | 548 kB 00:00 2024-04-24T23:00:20.771 INFO:teuthology.orchestra.run.smithi183.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 545 kB/s | 309 kB 00:00 2024-04-24T23:00:20.846 INFO:teuthology.orchestra.run.smithi183.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-04-24T23:00:20.938 INFO:teuthology.orchestra.run.smithi183.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-24T23:00:21.030 INFO:teuthology.orchestra.run.smithi183.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-04-24T23:00:21.056 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:00:21.092 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:21.130 INFO:teuthology.orchestra.run.smithi183.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-04-24T23:00:21.182 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:00:21.222 INFO:teuthology.orchestra.run.smithi183.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-04-24T23:00:21.247 INFO:teuthology.orchestra.run.smithi183.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-24T23:00:21.249 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:00:21.339 INFO:teuthology.orchestra.run.smithi183.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-04-24T23:00:21.398 INFO:teuthology.orchestra.run.smithi183.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-24T23:00:21.440 INFO:teuthology.orchestra.run.smithi183.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-04-24T23:00:21.498 INFO:teuthology.orchestra.run.smithi183.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-24T23:00:21.541 INFO:teuthology.orchestra.run.smithi183.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-24T23:00:21.608 INFO:teuthology.orchestra.run.smithi183.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-24T23:00:21.641 INFO:teuthology.orchestra.run.smithi183.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-24T23:00:21.718 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:00:21.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:21.816 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:21.838 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:00:21.933 INFO:teuthology.orchestra.run.smithi183.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 931 kB/s | 303 kB 00:00 2024-04-24T23:00:21.958 INFO:teuthology.orchestra.run.smithi183.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 111 kB/s | 35 kB 00:00 2024-04-24T23:00:22.050 INFO:teuthology.orchestra.run.smithi183.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-24T23:00:22.101 INFO:teuthology.orchestra.run.smithi183.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-24T23:00:22.159 INFO:teuthology.orchestra.run.smithi183.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-24T23:00:22.201 INFO:teuthology.orchestra.run.smithi183.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-24T23:00:22.260 INFO:teuthology.orchestra.run.smithi183.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-24T23:00:22.302 INFO:teuthology.orchestra.run.smithi183.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-24T23:00:22.360 INFO:teuthology.orchestra.run.smithi183.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-24T23:00:22.402 INFO:teuthology.orchestra.run.smithi183.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-24T23:00:22.503 INFO:teuthology.orchestra.run.smithi183.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-24T23:00:22.545 INFO:teuthology.orchestra.run.smithi183.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-04-24T23:00:22.595 INFO:teuthology.orchestra.run.smithi183.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 861 kB/s | 79 kB 00:00 2024-04-24T23:00:22.645 INFO:teuthology.orchestra.run.smithi183.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-04-24T23:00:22.695 INFO:teuthology.orchestra.run.smithi183.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-24T23:00:22.746 INFO:teuthology.orchestra.run.smithi183.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-24T23:00:22.796 INFO:teuthology.orchestra.run.smithi183.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-04-24T23:00:22.846 INFO:teuthology.orchestra.run.smithi183.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-24T23:00:22.896 INFO:teuthology.orchestra.run.smithi183.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-24T23:00:22.946 INFO:teuthology.orchestra.run.smithi183.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-24T23:00:22.997 INFO:teuthology.orchestra.run.smithi183.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-04-24T23:00:23.047 INFO:teuthology.orchestra.run.smithi183.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-24T23:00:23.059 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:23.060 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-04-24T23:00:23.060 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-04-24T23:00:23.060 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-04-24T23:00:23.060 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-04-24T23:00:23.060 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-04-24T23:00:23.060 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-04-24T23:00:23.060 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-04-24T23:00:23.060 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:23.098 INFO:teuthology.orchestra.run.smithi183.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-04-24T23:00:23.148 INFO:teuthology.orchestra.run.smithi183.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-24T23:00:23.198 INFO:teuthology.orchestra.run.smithi183.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 896 kB/s | 90 kB 00:00 2024-04-24T23:00:23.220 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:23.235 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:23.236 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:23.236 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:00:23.236 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:23.256 INFO:teuthology.orchestra.run.smithi183.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-24T23:00:23.290 INFO:teuthology.orchestra.run.smithi183.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-24T23:00:23.357 INFO:teuthology.orchestra.run.smithi183.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-24T23:00:23.416 INFO:teuthology.orchestra.run.smithi183.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-24T23:00:23.858 INFO:teuthology.orchestra.run.smithi183.stdout:(148/152): librados2-19.0.0-2445.gd2c45d97.el9. 6.8 MB/s | 3.4 MB 00:00 2024-04-24T23:00:24.058 INFO:teuthology.orchestra.run.smithi183.stdout:(149/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.9 MB/s | 3.1 MB 00:00 2024-04-24T23:00:24.116 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:24.142 INFO:teuthology.orchestra.run.smithi183.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.8 MB/s | 1.3 MB 00:00 2024-04-24T23:00:24.151 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:24.384 INFO:teuthology.orchestra.run.smithi183.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 2.4 MB 00:00 2024-04-24T23:00:24.694 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:24.733 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:24.738 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:24.759 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:24.759 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:24.759 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:00:24.759 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:24.759 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:24.759 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:24.832 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:24.832 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:00:24.832 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:00:24.832 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:24.864 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:24.983 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:25.006 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:25.006 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:25.006 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:00:25.006 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:25.006 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:25.006 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:25.443 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:25.465 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:25.465 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:25.465 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:00:25.465 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:25.465 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:25.465 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:25.581 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:00:25.630 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:00:25.684 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:00:25.867 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:00:25.919 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:00:25.996 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:00:26.083 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:00:26.214 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:00:26.252 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:00:26.307 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:00:26.336 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:00:26.752 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:26.777 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:26.808 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:00:26.870 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:00:26.908 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:27.932 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:27.968 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:28.934 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:28.960 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:29.061 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:29.233 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:00:29.271 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:29.294 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:29.294 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:29.294 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:00:29.294 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:29.294 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:29.294 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:29.338 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:29.349 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:29.446 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:00:29.494 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:00:30.388 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:00:30.416 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:30.438 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:30.439 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:30.439 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:00:30.439 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:30.439 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:30.439 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:30.483 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:30.505 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:30.505 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:30.505 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:00:30.505 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:30.551 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:30.573 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:30.573 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:00:30.573 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:30.573 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:30.876 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:30.899 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:30.899 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:30.899 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:00:30.899 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:30.899 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:30.899 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:34.616 INFO:teuthology.orchestra.run.smithi183.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-04-24T23:00:34.619 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:00:34.619 INFO:teuthology.orchestra.run.smithi183.stdout:Total 5.3 MB/s | 220 MB 00:41 2024-04-24T23:00:35.397 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:00:35.438 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:00:35.479 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:00:35.511 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:00:35.538 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:00:35.717 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:00:35.731 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-04-24T23:00:35.759 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:00:35.781 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:00:35.874 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-04-24T23:00:35.875 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-04-24T23:00:36.268 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:36.268 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-04-24T23:00:36.268 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-04-24T23:00:36.268 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-04-24T23:00:36.268 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-04-24T23:00:36.268 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-04-24T23:00:36.268 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-04-24T23:00:36.268 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-04-24T23:00:36.268 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:36.402 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:36.427 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:36.442 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:00:36.442 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:36.443 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:00:36.443 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:36.446 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:36.465 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:37.282 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:00:37.367 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:00:37.408 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:00:37.490 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:00:37.530 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:00:37.530 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:37.614 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:37.619 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-04-24T23:00:37.619 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-04-24T23:00:37.647 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:00:37.647 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:37.752 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:37.820 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:00:37.914 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:37.939 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:00:37.939 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:37.939 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:00:37.939 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:37.939 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:00:37.939 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:38.164 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:38.186 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:00:38.186 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:38.187 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:00:38.187 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:38.187 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:00:38.187 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:38.623 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:38.645 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:00:38.645 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:38.645 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:00:38.645 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:38.646 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:00:38.646 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:38.753 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:00:38.801 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:00:38.857 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:00:39.041 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:00:39.090 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:00:39.159 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:00:39.248 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:00:39.387 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:00:39.423 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:00:39.473 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:00:39.500 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:00:39.898 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:39.923 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:00:39.955 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:00:40.016 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:00:40.055 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:41.082 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:00:41.115 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:00:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:42.207 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:00:42.369 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:00:42.401 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:42.424 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:00:42.424 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:42.424 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:00:42.424 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:42.424 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:00:42.424 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:42.468 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:42.480 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:00:42.577 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:00:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:00:43.497 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-04-24T23:00:43.526 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:00:43.562 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:43.586 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:00:43.586 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:00:43.586 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:00:43.587 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:00:43.587 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:00:43.587 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:00:43.587 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:00:43.587 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:00:43.587 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:43.587 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:43.587 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:00:43.588 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:00:43.588 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:00:43.588 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:43.588 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:00:43.588 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:00:43.588 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:00:43.588 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:00:43.588 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:00:43.588 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:00:43.589 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:00:43.589 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:00:43.589 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:00:43.589 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:00:43.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:00:43.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:00:43.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:00:43.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:00:43.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:00:43.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:00:43.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:00:43.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:00:43.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:00:43.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:00:43.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:00:43.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:00:43.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:00:43.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:00:43.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:00:43.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:00:43.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:00:43.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:00:43.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:00:43.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:00:43.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:00:43.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:00:43.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:00:43.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:00:43.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:00:43.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:43.601 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:00:43.623 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:00:43.629 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:00:43.654 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:43.654 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:00:43.654 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:43.676 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:00:43.697 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:43.711 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:00:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:00:43.720 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:43.720 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:00:43.720 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:43.720 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:00:43.720 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:43.745 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:00:43.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:00:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:44.053 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:00:44.053 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:00:44.053 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:00:44.053 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:44.054 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:00:44.054 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:44.108 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:00:44.229 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:44.350 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:00:44.654 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:00:44.698 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:00:44.726 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:00:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:44.953 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:00:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:45.135 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:00:45.242 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:00:45.328 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:00:45.359 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:00:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:00:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:00:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:00:45.538 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:00:45.570 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:00:45.614 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:00:45.666 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.667 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.668 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:00:45.669 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:00:45.670 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:00:45.671 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:00:45.672 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:00:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:00:45.674 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:00:45.675 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:00:45.676 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:00:45.677 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:00:45.678 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:00:45.678 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:00:45.678 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-04-24T23:00:45.683 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:00:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:00:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:00:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:00:45.917 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:00:45.955 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:00:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:00:46.019 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:00:46.058 DEBUG:teuthology.parallel:result is None 2024-04-24T23:00:46.089 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:00:46.143 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:00:46.216 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:00:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:00:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:00:46.926 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:00:47.006 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:00:47.036 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:00:47.078 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:00:47.116 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:00:47.158 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:00:47.310 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:00:47.361 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:00:47.414 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:00:47.479 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:00:47.520 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:00:47.556 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:00:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:00:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:00:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:00:47.732 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:00:47.868 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:00:48.088 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:00:48.118 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:00:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:00:48.270 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:00:48.301 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:00:48.341 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:00:48.367 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:00:48.392 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:00:48.456 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:00:48.551 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:00:48.583 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:00:48.592 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:00:48.624 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:00:48.633 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:00:48.652 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:00:48.665 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:00:48.686 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:00:48.693 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:00:48.725 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:00:48.757 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:00:48.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:00:48.821 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:00:48.855 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:00:48.866 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:00:48.896 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:00:48.914 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:00:48.939 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:00:48.944 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:00:49.024 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:00:49.069 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:00:49.101 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:00:49.147 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:00:49.270 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:00:49.319 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:00:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:49.579 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:00:50.033 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:00:50.082 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:00:50.116 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:00:50.213 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:00:50.265 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:00:50.352 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:00:50.410 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:00:50.428 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:00:50.513 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:00:50.554 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:00:50.586 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:00:50.624 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:00:50.636 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:00:50.676 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:00:50.676 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:50.701 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:00:50.751 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:00:50.785 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:00:50.785 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:00:50.924 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:00:51.336 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:00:51.446 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:00:52.670 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:00:52.753 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:00:52.798 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:00:52.831 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:00:53.141 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:00:53.194 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:53.279 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:00:53.306 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:00:53.343 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:00:53.816 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:00:53.846 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:00:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:00:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:56.644 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:00:56.644 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:00:56.644 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:00:56.644 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:00:56.644 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:00:56.644 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:00:56.644 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:00:56.645 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:00:56.646 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:00:56.646 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:00:56.646 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:00:56.646 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:00:56.646 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:00:56.646 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:00:56.646 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:00:56.646 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:00:56.659 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:00:56.659 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:00:56.659 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:00:56.659 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:00:56.659 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:00:56.659 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:00:56.659 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:00:56.659 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:00:56.660 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:00:56.661 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:00:56.662 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:00:56.663 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:00:56.664 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:00:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:00:56.666 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:00:56.667 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:00:56.668 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:00:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:00:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:56.871 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:00:56.871 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:00:56.871 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:00:56.871 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:00:56.899 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:00:58.774 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:00:58.774 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:58.774 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-04-24T23:00:58.774 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.774 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.775 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:00:58.777 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:00:58.778 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:00:58.779 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.780 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:00:58.781 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:00:58.782 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:00:58.783 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:00:58.784 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:00:58.785 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:00:58.786 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:00:58.786 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:00:58.786 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:00:58.786 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:00:58.786 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-24T23:00:58.786 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-04-24T23:00:59.153 DEBUG:teuthology.parallel:result is None 2024-04-24T23:01:08.280 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:01:08.280 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-04-24T23:01:08.280 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-04-24T23:01:08.280 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-04-24T23:01:08.281 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-04-24T23:01:08.281 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-04-24T23:01:08.281 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-04-24T23:01:08.281 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-04-24T23:01:08.281 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:01:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:01:08.449 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:08.450 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:01:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:09.919 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:01:09.946 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:01:09.946 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:09.947 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:01:09.947 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:01:09.947 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:01:09.947 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:10.179 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:01:10.201 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:01:10.201 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:10.201 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:01:10.201 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:01:10.201 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:01:10.202 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:10.664 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:01:10.686 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:01:10.686 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:10.687 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:01:10.687 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:01:10.687 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:01:10.687 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:10.786 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:01:10.834 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:01:10.889 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:01:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:01:11.115 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:01:11.209 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:01:11.298 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:01:11.437 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:01:11.473 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:01:11.505 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:01:11.533 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:01:11.940 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:01:11.973 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:01:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:01:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:01:12.104 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:01:13.134 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:01:13.164 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:01:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:01:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:01:14.247 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:01:14.411 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:01:14.442 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:01:14.465 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:01:14.465 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:14.465 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:01:14.465 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:01:14.465 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:01:14.465 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:14.509 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:01:14.520 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:01:14.617 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:01:14.673 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:01:15.568 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:01:15.598 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:01:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:01:15.618 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:15.618 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:01:15.618 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:01:15.618 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:01:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:01:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:01:15.685 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:15.685 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:01:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:15.729 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:01:15.751 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:01:15.751 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:15.751 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:01:15.752 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:01:15.752 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:01:15.752 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:16.063 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:01:16.086 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:01:16.086 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:16.086 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:01:16.087 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:01:16.087 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:01:16.087 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:20.583 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:01:20.624 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:01:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:01:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:01:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:01:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:01:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:01:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:01:21.619 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:01:21.665 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:01:21.684 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:01:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:01:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:01:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:01:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:01:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:01:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:01:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:01:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:01:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:01:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:01:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:28.955 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:28.955 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:01:28.955 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:01:28.955 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:01:28.955 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:01:28.956 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:01:28.957 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:01:28.969 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:01:28.969 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:01:28.969 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:01:28.969 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:01:28.970 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:01:28.971 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:01:28.972 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:01:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:01:28.974 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:01:28.975 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:01:28.976 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:01:28.977 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:01:28.978 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:01:31.002 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:01:31.002 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:31.002 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-04-24T23:01:31.002 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.002 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.002 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:31.002 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:31.002 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.004 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:01:31.005 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.006 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:01:31.007 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.008 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:01:31.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:01:31.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:01:31.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:01:31.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:01:31.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:01:31.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:01:31.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:01:31.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:01:31.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:01:31.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:01:31.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:01:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.013 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.014 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:01:31.014 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:01:31.014 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:01:31.014 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:01:31.014 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:01:31.014 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:01:31.014 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:01:31.014 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:01:31.015 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:01:31.015 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-04-24T23:01:31.410 DEBUG:teuthology.parallel:result is None 2024-04-24T23:01:31.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:31.557 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:01:31.585 INFO:teuthology.orchestra.run.smithi102.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:31.586 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:31.586 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:01:31.588 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:31.733 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:01:31.758 INFO:teuthology.orchestra.run.smithi161.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:31.758 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:31.758 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:01:31.760 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:31.904 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:01:31.927 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:31.927 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:01:31.927 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:01:31.929 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T23:01:31.929 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:01:31.929 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:01:31.960 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T23:01:31.960 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:01:31.994 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T23:01:31.994 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:01:32.020 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T23:01:32.020 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:01:32.020 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:01:32.047 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:01:32.112 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T23:01:32.112 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:01:32.141 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:01:32.209 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T23:01:32.209 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:01:32.232 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:01:32.296 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T23:01:32.296 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:01:32.297 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:01:32.322 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:01:32.387 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T23:01:32.388 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:01:32.414 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:01:32.479 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T23:01:32.479 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:01:32.502 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:01:32.568 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T23:01:32.568 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:01:32.568 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:01:32.593 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:01:32.658 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T23:01:32.659 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:01:32.683 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:01:32.747 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T23:01:32.747 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:01:32.771 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:01:32.836 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T23:01:32.938 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'SLOW_OPS', 'slow request'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-24T23:01:32.939 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:32.939 INFO:tasks.cephadm:Cluster fsid is 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:01:32.939 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T23:01:32.939 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.102', 'mon.b': '172.21.15.161', 'mon.c': '172.21.15.183'} 2024-04-24T23:01:32.940 INFO:tasks.cephadm:First mon is mon.a on smithi102 2024-04-24T23:01:32.940 INFO:tasks.cephadm:First mgr is x 2024-04-24T23:01:32.940 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T23:01:32.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2024-04-24T23:01:32.968 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-04-24T23:01:32.993 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-04-24T23:01:33.017 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T23:01:33.017 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:33.165 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T23:01:33.312 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:33.314 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-24T23:01:33.314 DEBUG:teuthology.orchestra.run.smithi102:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:33.863 INFO:teuthology.orchestra.run.smithi102.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:01 /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:33.863 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:34.276 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:01 /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:34.276 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:34.684 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:01 /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:34.684 DEBUG:teuthology.orchestra.run.smithi102:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:34.702 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:34.720 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:01:34.756 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-24T23:01:34.756 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:01:34.759 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:01:34.763 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:01:36.449 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:01:36.487 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:01:36.535 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:02:08.664 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-24T23:02:08.664 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:02:08.664 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:02:08.664 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-04-24T23:02:08.664 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:02:08.664 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-04-24T23:02:08.664 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-24T23:02:29.017 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:02:29.017 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:02:29.017 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:02:29.017 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2024-04-24T23:02:29.017 INFO:teuthology.orchestra.run.smithi102.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:02:29.017 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:02:29.017 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:02:34.724 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-24T23:02:34.724 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:02:34.724 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:02:34.724 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-04-24T23:02:34.724 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:02:34.724 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-04-24T23:02:34.724 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-24T23:02:34.748 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2024-04-24T23:02:34.780 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-04-24T23:02:34.812 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-04-24T23:02:34.838 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2024-04-24T23:02:34.868 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-04-24T23:02:34.897 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-04-24T23:02:34.920 INFO:tasks.cephadm:Writing seed config... 2024-04-24T23:02:34.921 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T23:02:34.922 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T23:02:34.923 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T23:02:34.924 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T23:02:34.925 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T23:02:34.925 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-04-24T23:02:34.925 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T23:02:34.925 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T23:02:34.925 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T23:02:34.925 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-24T23:02:34.926 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:02:34.926 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T23:02:34.942 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 97eecd66-028e-11ef-bc93-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T23:02:34.943 DEBUG:teuthology.orchestra.run.smithi102:mon.a> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T23:02:34.985 DEBUG:teuthology.orchestra.run.smithi102:mgr.x> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T23:02:35.027 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T23:02:35.027 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid 97eecd66-028e-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.102 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:02:35.217 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:35.217 INFO:teuthology.orchestra.run.smithi102.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', '97eecd66-028e-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.102', '--skip-admin-label'] 2024-04-24T23:02:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T23:02:35.218 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying podman|docker is present... 2024-04-24T23:02:35.242 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:02:35.242 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying lvm2 is present... 2024-04-24T23:02:35.242 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying time synchronization is in place... 2024-04-24T23:02:35.250 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:02:35.250 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:02:35.256 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:02:35.256 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-04-24T23:02:35.262 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout enabled 2024-04-24T23:02:35.268 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout active 2024-04-24T23:02:35.269 INFO:teuthology.orchestra.run.smithi102.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:02:35.269 INFO:teuthology.orchestra.run.smithi102.stdout:Repeating the final host check... 2024-04-24T23:02:35.291 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:02:35.291 INFO:teuthology.orchestra.run.smithi102.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T23:02:35.291 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl is present 2024-04-24T23:02:35.292 INFO:teuthology.orchestra.run.smithi102.stdout:lvcreate is present 2024-04-24T23:02:35.298 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:02:35.298 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:02:35.304 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:02:35.304 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-04-24T23:02:35.310 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout enabled 2024-04-24T23:02:35.316 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout active 2024-04-24T23:02:35.316 INFO:teuthology.orchestra.run.smithi102.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:02:35.317 INFO:teuthology.orchestra.run.smithi102.stdout:Host looks OK 2024-04-24T23:02:35.317 INFO:teuthology.orchestra.run.smithi102.stdout:Cluster fsid: 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:35.317 INFO:teuthology.orchestra.run.smithi102.stdout:Acquiring lock 140371448005536 on /run/cephadm/97eecd66-028e-11ef-bc93-c7b262605968.lock 2024-04-24T23:02:35.317 INFO:teuthology.orchestra.run.smithi102.stdout:Lock 140371448005536 acquired on /run/cephadm/97eecd66-028e-11ef-bc93-c7b262605968.lock 2024-04-24T23:02:35.317 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying IP 172.21.15.102 port 3300 ... 2024-04-24T23:02:35.317 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying IP 172.21.15.102 port 6789 ... 2024-04-24T23:02:35.317 INFO:teuthology.orchestra.run.smithi102.stdout:Base mon IP(s) is [172.21.15.102:3300, 172.21.15.102:6789], mon addrv is [v2:172.21.15.102:3300,v1:172.21.15.102:6789] 2024-04-24T23:02:35.320 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.102 metric 100 2024-04-24T23:02:35.320 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.102 metric 100 2024-04-24T23:02:35.323 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T23:02:35.323 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-24T23:02:35.323 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-24T23:02:35.325 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T23:02:35.325 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T23:02:35.325 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:02:35.325 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T23:02:35.325 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout inet6 fe80::cd07:e13c:958c:c87a/64 scope link noprefixroute 2024-04-24T23:02:35.326 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:02:35.326 INFO:teuthology.orchestra.run.smithi102.stdout:Mon IP `172.21.15.102` is in CIDR network `172.21.0.0/20` 2024-04-24T23:02:35.327 INFO:teuthology.orchestra.run.smithi102.stdout:Mon IP `172.21.15.102` is in CIDR network `172.21.0.0/20` 2024-04-24T23:02:35.327 INFO:teuthology.orchestra.run.smithi102.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T23:02:35.327 INFO:teuthology.orchestra.run.smithi102.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T23:02:35.328 INFO:teuthology.orchestra.run.smithi102.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:02:35.796 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:02:35.797 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:02:35.797 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T23:02:35.797 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-24T23:02:35.797 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T23:02:35.797 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:02:35.797 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T23:02:36.701 INFO:teuthology.orchestra.run.smithi102.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:02:36.701 INFO:teuthology.orchestra.run.smithi102.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:02:36.702 INFO:teuthology.orchestra.run.smithi102.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T23:02:37.945 INFO:teuthology.orchestra.run.smithi102.stdout:stat: stdout 167 167 2024-04-24T23:02:37.945 INFO:teuthology.orchestra.run.smithi102.stdout:Creating initial keys... 2024-04-24T23:02:38.842 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-authtool: stdout AQCOjylm9yywDhAAxSv5RH2L+2YzCEUiyn0nJQ== 2024-04-24T23:02:39.883 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-authtool: stdout AQCPjylmHeiOCxAA2Mim2m1e6c7V7p8pS708CA== 2024-04-24T23:02:40.875 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-authtool: stdout AQCQjylmFbZ9CxAAnW/O04h7n/2OxxSTf8MOeA== 2024-04-24T23:02:40.876 INFO:teuthology.orchestra.run.smithi102.stdout:Creating initial monmap... 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool for a [v2:172.21.15.102:3300,v1:172.21.15.102:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout:setting min_mon_release = quincy 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: set fsid to 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:02:41.815 INFO:teuthology.orchestra.run.smithi102.stdout:Creating mon... 2024-04-24T23:02:51.211 INFO:teuthology.orchestra.run.smithi102.stdout:create mon.a on 2024-04-24T23:02:51.655 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T23:02:51.946 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T23:02:52.365 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-97eecd66-028e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-97eecd66-028e-11ef-bc93-c7b262605968.target. 2024-04-24T23:02:52.365 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-97eecd66-028e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-97eecd66-028e-11ef-bc93-c7b262605968.target. 2024-04-24T23:02:52.596 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.a 2024-04-24T23:02:52.596 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to reset failed state of unit ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T23:02:52.764 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-97eecd66-028e-11ef-bc93-c7b262605968.target.wants/ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-97eecd66-028e-11ef-bc93-c7b262605968@.service. 2024-04-24T23:02:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:02:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-04-24T23:02:53.306 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:02:53.306 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-04-24T23:02:53.306 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-04-24T23:02:53.306 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:02:53.306 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mon to start... 2024-04-24T23:02:53.306 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mon... 2024-04-24T23:02:54.392 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:54 smithi102 ceph-mon[28840]: from='client.? 172.21.15.102:0/3557779318' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T23:02:54.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T23:02:54.401 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout id: 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:54.401 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T23:02:54.401 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:54.401 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout services: 2024-04-24T23:02:54.401 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.596783s) 2024-04-24T23:02:54.401 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T23:02:54.401 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout data: 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.767+0000 7f4e2ae58700 1 Processor -- start 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.767+0000 7f4e2ae58700 1 -- start start 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.767+0000 7f4e2ae58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e24107120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.767+0000 7f4e2ae58700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241076f0 con 0x7f4e24106d00 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.768+0000 7f4e28bf4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e24107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:54.402 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.768+0000 7f4e28bf4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e24107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38814/0 (socket says 172.21.15.102:38814) 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.768+0000 7f4e28bf4700 1 -- 172.21.15.102:0/3716514457 learned_addr learned my addr 172.21.15.102:0/3716514457 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.768+0000 7f4e28bf4700 1 -- 172.21.15.102:0/3716514457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e24107f00 con 0x7f4e24106d00 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.768+0000 7f4e28bf4700 1 --2- 172.21.15.102:0/3716514457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e24107120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4e200098a0 tx=0x7f4e20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2cacdeff605228e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.768+0000 7f4e1b7fe700 1 -- 172.21.15.102:0/3716514457 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e20004160 con 0x7f4e24106d00 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.768+0000 7f4e1b7fe700 1 -- 172.21.15.102:0/3716514457 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e20004300 con 0x7f4e24106d00 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.768+0000 7f4e1b7fe700 1 -- 172.21.15.102:0/3716514457 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e200036a0 con 0x7f4e24106d00 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.769+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3716514457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 msgr2=0x7f4e24107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.769+0000 7f4e2ae58700 1 --2- 172.21.15.102:0/3716514457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e24107120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4e200098a0 tx=0x7f4e20009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.769+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3716514457 shutdown_connections 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.769+0000 7f4e2ae58700 1 --2- 172.21.15.102:0/3716514457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e24107120 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:54.403 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.769+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3716514457 >> 172.21.15.102:0/3716514457 conn(0x7f4e24102280 msgr2=0x7f4e241046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.769+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3716514457 shutdown_connections 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.769+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3716514457 wait complete. 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e2ae58700 1 Processor -- start 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e2ae58700 1 -- start start 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e2ae58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e2407bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e2ae58700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24108430 con 0x7f4e24106d00 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e28bf4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e2407bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e28bf4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e2407bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38826/0 (socket says 172.21.15.102:38826) 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e28bf4700 1 -- 172.21.15.102:0/3557779318 learned_addr learned my addr 172.21.15.102:0/3557779318 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e28bf4700 1 -- 172.21.15.102:0/3557779318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2419e170 con 0x7f4e24106d00 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e28bf4700 1 --2- 172.21.15.102:0/3557779318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e2407bab0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4e20000c00 tx=0x7f4e20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.770+0000 7f4e19ffb700 1 -- 172.21.15.102:0/3557779318 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e20003db0 con 0x7f4e24106d00 2024-04-24T23:02:54.404 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.771+0000 7f4e19ffb700 1 -- 172.21.15.102:0/3557779318 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e20011630 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.771+0000 7f4e19ffb700 1 -- 172.21.15.102:0/3557779318 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e20011950 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.771+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3557779318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2419e340 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.771+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3557779318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2419e760 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.771+0000 7f4e19ffb700 1 -- 172.21.15.102:0/3557779318 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e20011af0 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.771+0000 7f4e19ffb700 1 -- 172.21.15.102:0/3557779318 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e2001c450 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.772+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3557779318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e24063890 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.774+0000 7f4e19ffb700 1 -- 172.21.15.102:0/3557779318 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4e20032020 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.845+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3557779318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4e2402de70 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.846+0000 7f4e19ffb700 1 -- 172.21.15.102:0/3557779318 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f4e20004260 con 0x7f4e24106d00 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.846+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3557779318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 msgr2=0x7f4e2407bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.846+0000 7f4e2ae58700 1 --2- 172.21.15.102:0/3557779318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e2407bab0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4e20000c00 tx=0x7f4e20004750 comp rx=0 tx=0).stop 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.846+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3557779318 shutdown_connections 2024-04-24T23:02:54.405 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.846+0000 7f4e2ae58700 1 --2- 172.21.15.102:0/3557779318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e24106d00 0x7f4e2407bab0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:54.406 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.846+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3557779318 >> 172.21.15.102:0/3557779318 conn(0x7f4e24102280 msgr2=0x7f4e24190fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:54.406 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.846+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3557779318 shutdown_connections 2024-04-24T23:02:54.406 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:53.846+0000 7f4e2ae58700 1 -- 172.21.15.102:0/3557779318 wait complete. 2024-04-24T23:02:54.406 INFO:teuthology.orchestra.run.smithi102.stdout:mon is available 2024-04-24T23:02:54.406 INFO:teuthology.orchestra.run.smithi102.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T23:02:55.510 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:55.510 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:02:55.510 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout fsid = 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:55.510 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.102:3300,v1:172.21.15.102:6789] 2024-04-24T23:02:55.510 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.850+0000 7f02fbcb5700 1 Processor -- start 2024-04-24T23:02:55.511 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.850+0000 7f02fbcb5700 1 -- start start 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.850+0000 7f02fbcb5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f410ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.850+0000 7f02fbcb5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f02f407be50 con 0x7f02f4108890 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.850+0000 7f02f9a51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f410ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.850+0000 7f02f9a51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f410ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38840/0 (socket says 172.21.15.102:38840) 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.850+0000 7f02f9a51700 1 -- 172.21.15.102:0/3462431210 learned_addr learned my addr 172.21.15.102:0/3462431210 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.850+0000 7f02f9a51700 1 -- 172.21.15.102:0/3462431210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f410b1c0 con 0x7f02f4108890 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02f9a51700 1 --2- 172.21.15.102:0/3462431210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f410ac80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f02f00098a0 tx=0x7f02f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bd1c1b700ffd93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02f8a4f700 1 -- 172.21.15.102:0/3462431210 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f0004160 con 0x7f02f4108890 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02f8a4f700 1 -- 172.21.15.102:0/3462431210 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f0004300 con 0x7f02f4108890 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02f8a4f700 1 -- 172.21.15.102:0/3462431210 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f00036a0 con 0x7f02f4108890 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02fbcb5700 1 -- 172.21.15.102:0/3462431210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 msgr2=0x7f02f410ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02fbcb5700 1 --2- 172.21.15.102:0/3462431210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f410ac80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f02f00098a0 tx=0x7f02f0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02fbcb5700 1 -- 172.21.15.102:0/3462431210 shutdown_connections 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02fbcb5700 1 --2- 172.21.15.102:0/3462431210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f410ac80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02fbcb5700 1 -- 172.21.15.102:0/3462431210 >> 172.21.15.102:0/3462431210 conn(0x7f02f4102240 msgr2=0x7f02f41046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02fbcb5700 1 -- 172.21.15.102:0/3462431210 shutdown_connections 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.851+0000 7f02fbcb5700 1 -- 172.21.15.102:0/3462431210 wait complete. 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.852+0000 7f02fbcb5700 1 Processor -- start 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.852+0000 7f02fbcb5700 1 -- start start 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.852+0000 7f02fbcb5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.852+0000 7f02fbcb5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f02f410b630 con 0x7f02f4108890 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.852+0000 7f02f9a51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.852+0000 7f02f9a51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38842/0 (socket says 172.21.15.102:38842) 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.852+0000 7f02f9a51700 1 -- 172.21.15.102:0/1596380912 learned_addr learned my addr 172.21.15.102:0/1596380912 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.853+0000 7f02f9a51700 1 -- 172.21.15.102:0/1596380912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f419b970 con 0x7f02f4108890 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.853+0000 7f02f9a51700 1 --2- 172.21.15.102:0/1596380912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f419b430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f02f0000c00 tx=0x7f02f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.853+0000 7f02eaffd700 1 -- 172.21.15.102:0/1596380912 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f0003db0 con 0x7f02f4108890 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.853+0000 7f02eaffd700 1 -- 172.21.15.102:0/1596380912 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f0011630 con 0x7f02f4108890 2024-04-24T23:02:55.513 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.853+0000 7f02fbcb5700 1 -- 172.21.15.102:0/1596380912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f419bc00 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.853+0000 7f02eaffd700 1 -- 172.21.15.102:0/1596380912 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f0011950 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.853+0000 7f02fbcb5700 1 -- 172.21.15.102:0/1596380912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f419e8f0 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.853+0000 7f02eaffd700 1 -- 172.21.15.102:0/1596380912 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02f0011af0 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.853+0000 7f02eaffd700 1 -- 172.21.15.102:0/1596380912 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f02f001c450 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.854+0000 7f02fbcb5700 1 -- 172.21.15.102:0/1596380912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f4063890 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.856+0000 7f02eaffd700 1 -- 172.21.15.102:0/1596380912 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f02f0032020 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.926+0000 7f02fbcb5700 1 -- 172.21.15.102:0/1596380912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f02f402dcb0 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.929+0000 7f02eaffd700 1 -- 172.21.15.102:0/1596380912 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+456 (secure 0 0 0) 0x7f02f0004260 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.929+0000 7f02eaffd700 1 -- 172.21.15.102:0/1596380912 <== mon.0 v2:172.21.15.102:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f02f001c650 con 0x7f02f4108890 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.930+0000 7f02fbcb5700 1 -- 172.21.15.102:0/1596380912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 msgr2=0x7f02f419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.930+0000 7f02fbcb5700 1 --2- 172.21.15.102:0/1596380912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f419b430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f02f0000c00 tx=0x7f02f0004750 comp rx=0 tx=0).stop 2024-04-24T23:02:55.514 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.930+0000 7f02fbcb5700 1 -- 172.21.15.102:0/1596380912 shutdown_connections 2024-04-24T23:02:55.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.930+0000 7f02fbcb5700 1 --2- 172.21.15.102:0/1596380912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4108890 0x7f02f419b430 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:55.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.930+0000 7f02fbcb5700 1 -- 172.21.15.102:0/1596380912 >> 172.21.15.102:0/1596380912 conn(0x7f02f4102240 msgr2=0x7f02f41045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:55.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.930+0000 7f02fbcb5700 1 -- 172.21.15.102:0/1596380912 shutdown_connections 2024-04-24T23:02:55.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:54.930+0000 7f02fbcb5700 1 -- 172.21.15.102:0/1596380912 wait complete. 2024-04-24T23:02:55.515 INFO:teuthology.orchestra.run.smithi102.stdout:Generating new minimal ceph.conf... 2024-04-24T23:02:55.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:55 smithi102 ceph-mon[28840]: from='client.? 172.21.15.102:0/1596380912' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:02:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:55 smithi102 ceph-mon[28840]: from='client.? 172.21.15.102:0/1596380912' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T23:02:56.550 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:56 smithi102 ceph-mon[28840]: from='client.? 172.21.15.102:0/1567227868' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:02:56.557 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.891+0000 7fe8e9500700 1 Processor -- start 2024-04-24T23:02:56.557 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.891+0000 7fe8e9500700 1 -- start start 2024-04-24T23:02:56.557 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.892+0000 7fe8e9500700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e410acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.557 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.892+0000 7fe8e9500700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e407be70 con 0x7fe8e41088c0 2024-04-24T23:02:56.557 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.892+0000 7fe8e2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e410acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.557 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.892+0000 7fe8e2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e410acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38846/0 (socket says 172.21.15.102:38846) 2024-04-24T23:02:56.557 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.892+0000 7fe8e2ffd700 1 -- 172.21.15.102:0/100053580 learned_addr learned my addr 172.21.15.102:0/100053580 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:02:56.557 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.892+0000 7fe8e2ffd700 1 -- 172.21.15.102:0/100053580 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e410b1f0 con 0x7fe8e41088c0 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.892+0000 7fe8e2ffd700 1 --2- 172.21.15.102:0/100053580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e410acb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40098a0 tx=0x7fe8d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fe0863a7bd2098 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e1ffb700 1 -- 172.21.15.102:0/100053580 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d4004160 con 0x7fe8e41088c0 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e1ffb700 1 -- 172.21.15.102:0/100053580 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe8d4004300 con 0x7fe8e41088c0 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e1ffb700 1 -- 172.21.15.102:0/100053580 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d4011a80 con 0x7fe8e41088c0 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e9500700 1 -- 172.21.15.102:0/100053580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 msgr2=0x7fe8e410acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e9500700 1 --2- 172.21.15.102:0/100053580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e410acb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40098a0 tx=0x7fe8d4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e9500700 1 -- 172.21.15.102:0/100053580 shutdown_connections 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e9500700 1 --2- 172.21.15.102:0/100053580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e410acb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e9500700 1 -- 172.21.15.102:0/100053580 >> 172.21.15.102:0/100053580 conn(0x7fe8e41022b0 msgr2=0x7fe8e41046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e9500700 1 -- 172.21.15.102:0/100053580 shutdown_connections 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.893+0000 7fe8e9500700 1 -- 172.21.15.102:0/100053580 wait complete. 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.894+0000 7fe8e9500700 1 Processor -- start 2024-04-24T23:02:56.558 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.894+0000 7fe8e9500700 1 -- start start 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.894+0000 7fe8e9500700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.894+0000 7fe8e9500700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e410c0d0 con 0x7fe8e41088c0 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.894+0000 7fe8e2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.894+0000 7fe8e2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e419bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38848/0 (socket says 172.21.15.102:38848) 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.894+0000 7fe8e2ffd700 1 -- 172.21.15.102:0/1567227868 learned_addr learned my addr 172.21.15.102:0/1567227868 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.894+0000 7fe8e2ffd700 1 -- 172.21.15.102:0/1567227868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e419c210 con 0x7fe8e41088c0 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.895+0000 7fe8e2ffd700 1 --2- 172.21.15.102:0/1567227868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e419bcd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4011a50 tx=0x7fe8d4004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.895+0000 7fe8dbfff700 1 -- 172.21.15.102:0/1567227868 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d40039f0 con 0x7fe8e41088c0 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.895+0000 7fe8dbfff700 1 -- 172.21.15.102:0/1567227868 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe8d4003b90 con 0x7fe8e41088c0 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.895+0000 7fe8e9500700 1 -- 172.21.15.102:0/1567227868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e419c4a0 con 0x7fe8e41088c0 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.895+0000 7fe8dbfff700 1 -- 172.21.15.102:0/1567227868 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d401a600 con 0x7fe8e41088c0 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.895+0000 7fe8e9500700 1 -- 172.21.15.102:0/1567227868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e419f180 con 0x7fe8e41088c0 2024-04-24T23:02:56.559 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.895+0000 7fe8dbfff700 1 -- 172.21.15.102:0/1567227868 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8d401acb0 con 0x7fe8e41088c0 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.895+0000 7fe8dbfff700 1 -- 172.21.15.102:0/1567227868 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe8d4019700 con 0x7fe8e41088c0 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.896+0000 7fe8e9500700 1 -- 172.21.15.102:0/1567227868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e4063890 con 0x7fe8e41088c0 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.898+0000 7fe8dbfff700 1 -- 172.21.15.102:0/1567227868 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe8d4024a40 con 0x7fe8e41088c0 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.965+0000 7fe8e9500700 1 -- 172.21.15.102:0/1567227868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe8e419f350 con 0x7fe8e41088c0 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.965+0000 7fe8dbfff700 1 -- 172.21.15.102:0/1567227868 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fe8d4024440 con 0x7fe8e41088c0 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.966+0000 7fe8e9500700 1 -- 172.21.15.102:0/1567227868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 msgr2=0x7fe8e419bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.966+0000 7fe8e9500700 1 --2- 172.21.15.102:0/1567227868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e419bcd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4011a50 tx=0x7fe8d4004290 comp rx=0 tx=0).stop 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.966+0000 7fe8e9500700 1 -- 172.21.15.102:0/1567227868 shutdown_connections 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.966+0000 7fe8e9500700 1 --2- 172.21.15.102:0/1567227868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e41088c0 0x7fe8e419bcd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.966+0000 7fe8e9500700 1 -- 172.21.15.102:0/1567227868 >> 172.21.15.102:0/1567227868 conn(0x7fe8e41022b0 msgr2=0x7fe8e4102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.966+0000 7fe8e9500700 1 -- 172.21.15.102:0/1567227868 shutdown_connections 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:55.966+0000 7fe8e9500700 1 -- 172.21.15.102:0/1567227868 wait complete. 2024-04-24T23:02:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:Restarting the monitor... 2024-04-24T23:02:56.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:56 smithi102 systemd[1]: Stopping Ceph mon.a for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:02:57.192 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:56 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a[28836]: 2024-04-24T23:02:56.836+0000 7fa2b0d21700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:02:57.192 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:56 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a[28836]: 2024-04-24T23:02:56.836+0000 7fa2b0d21700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T23:02:57.193 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:56 smithi102 podman[29087]: 2024-04-24 23:02:56.949560846 +0000 UTC m=+0.231185639 container died 045981c37e824247df55fd3984ec8c5c75584c4a867a51462a3892cf8ecfcba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T23:02:58.473 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:58 smithi102 podman[29087]: 2024-04-24 23:02:58.157027232 +0000 UTC m=+1.438652022 container cleanup 045981c37e824247df55fd3984ec8c5c75584c4a867a51462a3892cf8ecfcba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T23:02:58.473 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:58 smithi102 bash[29087]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a 2024-04-24T23:02:58.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:58 smithi102 podman[29099]: 2024-04-24 23:02:58.688920064 +0000 UTC m=+1.744211937 container remove 045981c37e824247df55fd3984ec8c5c75584c4a867a51462a3892cf8ecfcba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T23:02:59.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:58 smithi102 systemd[1]: ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T23:02:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:58 smithi102 systemd[1]: Stopped Ceph mon.a for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:02:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:58 smithi102 systemd[1]: Starting Ceph mon.a for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:02:59.407 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 podman[29197]: 2024-04-24 23:02:59.055603008 +0000 UTC m=+0.018347868 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:02:59.407 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 podman[29197]: 2024-04-24 23:02:59.156519936 +0000 UTC m=+0.119264764 container create 4c8285e9b32c4e37c23d8aa1ecff5baa73383cd6f451a57b77de1fe878ecd948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T23:02:59.407 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 podman[29197]: 2024-04-24 23:02:59.375936769 +0000 UTC m=+0.338681618 container init 4c8285e9b32c4e37c23d8aa1ecff5baa73383cd6f451a57b77de1fe878ecd948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-24T23:02:59.407 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 podman[29197]: 2024-04-24 23:02:59.379574112 +0000 UTC m=+0.342318945 container start 4c8285e9b32c4e37c23d8aa1ecff5baa73383cd6f451a57b77de1fe878ecd948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:02:59.407 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:02:59.407 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:02:59.407 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: pidfile_write: ignore empty --pid-file 2024-04-24T23:02:59.407 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: load: jerasure load: lrc 2024-04-24T23:02:59.457 INFO:teuthology.orchestra.run.smithi102.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T23:02:59.703 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:02:59.704 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Git sha 0 2024-04-24T23:02:59.704 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:02:59.704 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: DB SUMMARY 2024-04-24T23:02:59.704 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: DB Session ID: 1UDOL6C3FQG4PG0J6OKZ 2024-04-24T23:02:59.704 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:02:59.704 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105830 ; 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.env: 0x55ed739e4c60 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.info_log: 0x55ed75387aa0 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.statistics: (nil) 2024-04-24T23:02:59.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.use_fsync: 0 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.db_log_dir: 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.wal_dir: 2024-04-24T23:02:59.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:02:59.707 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:02:59.707 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:02:59.707 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.write_buffer_manager: 0x55ed7535a1e0 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:02:59.709 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.unordered_write: 0 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.row_cache: None 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.wal_filter: None 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.wal_compression: 0 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:02:59.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:02:59.711 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:02:59.711 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:02:59.711 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:02:59.711 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:02:59.711 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:02:59.711 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_open_files: -1 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:02:59.712 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Compression algorithms supported: 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: kZSTD supported: 0 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:02:59.713 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.merge_operator: 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_filter: None 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ed75387ba0) 2024-04-24T23:02:59.714 INFO:journalctl@ceph.mon.a.smithi102.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: index_type: 0 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: data_block_index_type: 0 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: index_shortening: 1 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: checksum: 4 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: no_block_cache: 0 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_cache: 0x55ed7538b090 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_cache_options: 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: capacity : 536870912 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: num_shard_bits : 4 2024-04-24T23:02:59.715 INFO:journalctl@ceph.mon.a.smithi102.stdout: strict_capacity_limit : 0 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_cache_compressed: (nil) 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: persistent_cache: (nil) 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_size: 4096 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_size_deviation: 10 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_restart_interval: 16 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: index_block_restart_interval: 1 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: metadata_block_size: 4096 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: partition_filters: 0 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: use_delta_encoding: 1 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: filter_policy: bloomfilter 2024-04-24T23:02:59.716 INFO:journalctl@ceph.mon.a.smithi102.stdout: whole_key_filtering: 1 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout: verify_compression: 0 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout: format_version: 5 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout: enable_index_compression: 1 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_align: 0 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout: max_auto_readahead_size: 262144 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout: prepopulate_block_cache: 0 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression: NoCompression 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:02:59.717 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.num_levels: 7 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:02:59.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:02:59.719 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:02:59.720 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:02:59.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.ttl: 2592000 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:02:59.722 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 097b118c-16a9-42f7-bcbb-365ed14ed9ba 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999779408219, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999779409169, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98743, "index_size": 297, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14421, "raw_average_key_size": 49, "raw_value_size": 90934, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713999779, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "097b118c-16a9-42f7-bcbb-365ed14ed9ba", "db_session_id": "1UDOL6C3FQG4PG0J6OKZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999779409250, "job": 1, "event": "recovery_finished"} 2024-04-24T23:02:59.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ed7547c000 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: DB pointer 0x55ed75466000 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** DB Stats ** 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-04-24T23:02:59.724 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** Compaction Stats [default] ** 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: L0 2/0 100.29 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 166.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: Sum 2/0 100.29 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 166.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 166.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** Compaction Stats [default] ** 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 166.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:02:59.725 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: Cumulative compaction: 0.00 GB write, 31.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: Interval compaction: 0.00 GB write, 31.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: Block cache BinnedLRUCache@0x55ed7538b090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: starting mon.a rank 0 at public addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] at bind addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mon.a@-1(???) e1 preinit fsid 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mon.a@-1(???).mds e1 new map 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mon.a@-1(???).mds e1 print_map 2024-04-24T23:02:59.726 INFO:journalctl@ceph.mon.a.smithi102.stdout: e1 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout: legacy client fscid: -1 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout: No filesystems configured 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 bash[29197]: 4c8285e9b32c4e37c23d8aa1ecff5baa73383cd6f451a57b77de1fe878ecd948 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 systemd[1]: Started Ceph mon.a for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: monmap epoch 1 2024-04-24T23:02:59.727 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: fsid 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:59.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: last_changed 2024-04-24T23:02:41.198045+0000 2024-04-24T23:02:59.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: created 2024-04-24T23:02:41.198045+0000 2024-04-24T23:02:59.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: min_mon_release 19 (squid) 2024-04-24T23:02:59.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: election_strategy: 1 2024-04-24T23:02:59.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-04-24T23:02:59.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: fsmap 2024-04-24T23:02:59.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:02:59.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:02:59 smithi102 ceph-mon[29216]: mgrmap e1: no daemons active 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.891+0000 7fafa0938700 1 Processor -- start 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.891+0000 7fafa0938700 1 -- start start 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.891+0000 7fafa0938700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.891+0000 7fafa0938700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c108770 con 0x7faf9c107e10 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.892+0000 7faf9a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.892+0000 7faf9a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54374/0 (socket says 172.21.15.102:54374) 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.892+0000 7faf9a59c700 1 -- 172.21.15.102:0/2360536368 learned_addr learned my addr 172.21.15.102:0/2360536368 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.892+0000 7faf9a59c700 1 -- 172.21.15.102:0/2360536368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9c108f70 con 0x7faf9c107e10 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.892+0000 7faf9a59c700 1 --2- 172.21.15.102:0/2360536368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c108230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0098a0 tx=0x7faf8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde14b972cd13041 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.892+0000 7faf9959a700 1 -- 172.21.15.102:0/2360536368 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf8c004160 con 0x7faf9c107e10 2024-04-24T23:03:00.565 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.892+0000 7faf9959a700 1 -- 172.21.15.102:0/2360536368 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7faf8c004300 con 0x7faf9c107e10 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.892+0000 7faf9959a700 1 -- 172.21.15.102:0/2360536368 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf8c011a80 con 0x7faf9c107e10 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.893+0000 7fafa0938700 1 -- 172.21.15.102:0/2360536368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 msgr2=0x7faf9c108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.893+0000 7fafa0938700 1 --2- 172.21.15.102:0/2360536368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c108230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0098a0 tx=0x7faf8c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.893+0000 7fafa0938700 1 -- 172.21.15.102:0/2360536368 shutdown_connections 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.893+0000 7fafa0938700 1 --2- 172.21.15.102:0/2360536368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c108230 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.893+0000 7fafa0938700 1 -- 172.21.15.102:0/2360536368 >> 172.21.15.102:0/2360536368 conn(0x7faf9c103370 msgr2=0x7faf9c1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.893+0000 7fafa0938700 1 -- 172.21.15.102:0/2360536368 shutdown_connections 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.893+0000 7fafa0938700 1 -- 172.21.15.102:0/2360536368 wait complete. 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.894+0000 7fafa0938700 1 Processor -- start 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.894+0000 7fafa0938700 1 -- start start 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.894+0000 7fafa0938700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.894+0000 7fafa0938700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c10c230 con 0x7faf9c107e10 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.894+0000 7faf9a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:00.566 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.894+0000 7faf9a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54386/0 (socket says 172.21.15.102:54386) 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.894+0000 7faf9a59c700 1 -- 172.21.15.102:0/1206142118 learned_addr learned my addr 172.21.15.102:0/1206142118 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.894+0000 7faf9a59c700 1 -- 172.21.15.102:0/1206142118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9c19c210 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.895+0000 7faf9a59c700 1 --2- 172.21.15.102:0/1206142118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c19bcd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faf8c006af0 tx=0x7faf8c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.895+0000 7faf8b7fe700 1 -- 172.21.15.102:0/1206142118 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf8c003930 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.895+0000 7faf8b7fe700 1 -- 172.21.15.102:0/1206142118 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7faf8c003ad0 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.895+0000 7fafa0938700 1 -- 172.21.15.102:0/1206142118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9c19c4a0 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.895+0000 7faf8b7fe700 1 -- 172.21.15.102:0/1206142118 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf8c0195e0 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.895+0000 7fafa0938700 1 -- 172.21.15.102:0/1206142118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9c19f180 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.895+0000 7faf8b7fe700 1 -- 172.21.15.102:0/1206142118 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faf8c019cb0 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.895+0000 7faf8b7fe700 1 -- 172.21.15.102:0/1206142118 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faf8c024730 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.896+0000 7fafa0938700 1 -- 172.21.15.102:0/1206142118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf9c063890 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.898+0000 7faf8b7fe700 1 -- 172.21.15.102:0/1206142118 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7faf8c043dd0 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.968+0000 7fafa0938700 1 -- 172.21.15.102:0/1206142118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7faf9c19f350 con 0x7faf9c107e10 2024-04-24T23:03:00.567 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.970+0000 7faf8b7fe700 1 -- 172.21.15.102:0/1206142118 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7faf8c032020 con 0x7faf9c107e10 2024-04-24T23:03:00.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.972+0000 7fafa0938700 1 -- 172.21.15.102:0/1206142118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 msgr2=0x7faf9c19bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:00.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.972+0000 7fafa0938700 1 --2- 172.21.15.102:0/1206142118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c19bcd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faf8c006af0 tx=0x7faf8c011b70 comp rx=0 tx=0).stop 2024-04-24T23:03:00.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.972+0000 7fafa0938700 1 -- 172.21.15.102:0/1206142118 shutdown_connections 2024-04-24T23:03:00.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.972+0000 7fafa0938700 1 --2- 172.21.15.102:0/1206142118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9c107e10 0x7faf9c19bcd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:00.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.972+0000 7fafa0938700 1 -- 172.21.15.102:0/1206142118 >> 172.21.15.102:0/1206142118 conn(0x7faf9c103370 msgr2=0x7faf9c103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:00.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.972+0000 7fafa0938700 1 -- 172.21.15.102:0/1206142118 shutdown_connections 2024-04-24T23:03:00.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:02:59.972+0000 7fafa0938700 1 -- 172.21.15.102:0/1206142118 wait complete. 2024-04-24T23:03:00.568 INFO:teuthology.orchestra.run.smithi102.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T23:03:00.591 INFO:teuthology.orchestra.run.smithi102.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:00.591 INFO:teuthology.orchestra.run.smithi102.stdout:Creating mgr... 2024-04-24T23:03:00.591 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T23:03:00.591 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T23:03:00.839 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.x 2024-04-24T23:03:00.839 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to reset failed state of unit ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T23:03:00.987 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-97eecd66-028e-11ef-bc93-c7b262605968.target.wants/ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-97eecd66-028e-11ef-bc93-c7b262605968@.service. 2024-04-24T23:03:01.222 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:00 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1206142118' entity='client.admin' 2024-04-24T23:03:01.222 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 systemd[1]: Starting Ceph mgr.x for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:03:01.495 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 podman[29444]: 2024-04-24 23:03:01.123966209 +0000 UTC m=+0.017903136 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:03:01.496 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 podman[29444]: 2024-04-24 23:03:01.234067915 +0000 UTC m=+0.128004822 container create 78ac8b135b5e5acb061a03d8b5063b011ee1d01c565d7c90e6c3b7dc010b491d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T23:03:01.496 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 podman[29444]: 2024-04-24 23:03:01.436498021 +0000 UTC m=+0.330434948 container init 78ac8b135b5e5acb061a03d8b5063b011ee1d01c565d7c90e6c3b7dc010b491d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:03:01.496 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 podman[29444]: 2024-04-24 23:03:01.438986312 +0000 UTC m=+0.332923220 container start 78ac8b135b5e5acb061a03d8b5063b011ee1d01c565d7c90e6c3b7dc010b491d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T23:03:01.496 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 bash[29444]: 78ac8b135b5e5acb061a03d8b5063b011ee1d01c565d7c90e6c3b7dc010b491d 2024-04-24T23:03:01.509 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:03:01.509 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-04-24T23:03:01.517 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:03:01.517 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-04-24T23:03:01.517 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-04-24T23:03:01.517 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:03:01.524 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:03:01.525 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-04-24T23:03:01.532 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:03:01.532 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-04-24T23:03:01.532 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-04-24T23:03:01.532 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T23:03:01.532 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr to start... 2024-04-24T23:03:01.532 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr... 2024-04-24T23:03:01.783 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 systemd[1]: Started Ceph mgr.x for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:03:01.783 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:01.595+0000 7f1d61028200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:03:02.071 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:01.784+0000 7f1d61028200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:03:02.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:01.887+0000 7f1d61028200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:03:02.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:02 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1863385259' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:03:02.403 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:02 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:02.214+0000 7f1d61028200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:03:02.403 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:02 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:02.300+0000 7f1d61028200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:03:02.706 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:02.709 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:02.709 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:03:02.709 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:03:02.709 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:03:02.709 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:03:02.709 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:03:02.709 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:02.710 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:03:02.711 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:03:02.711 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:02.711 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:03:02.711 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:02.711 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:03:02.711 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:03:02.711 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:03:02.720 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:03:02.720 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:03:02.720 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:03:02.720 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:02.720 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:03:02.720 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:03:02.720 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:03:02.720 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:03:02.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:02.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:02:53.253856+0000", 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.997+0000 7f2f9ef2c700 1 Processor -- start 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.997+0000 7f2f9ef2c700 1 -- start start 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.997+0000 7f2f9ef2c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f981070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.997+0000 7f2f9ef2c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98107690 con 0x7f2f98106ca0 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.997+0000 7f2f9ccc8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f981070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.997+0000 7f2f9ccc8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f981070c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54412/0 (socket says 172.21.15.102:54412) 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.998+0000 7f2f9ccc8700 1 -- 172.21.15.102:0/243526836 learned_addr learned my addr 172.21.15.102:0/243526836 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.998+0000 7f2f9ccc8700 1 -- 172.21.15.102:0/243526836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f98107ea0 con 0x7f2f98106ca0 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.998+0000 7f2f9ccc8700 1 --2- 172.21.15.102:0/243526836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f981070c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0098a0 tx=0x7f2f8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb50be68d6ca88dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:02.723 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.998+0000 7f2f977fe700 1 -- 172.21.15.102:0/243526836 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f8c004160 con 0x7f2f98106ca0 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.998+0000 7f2f977fe700 1 -- 172.21.15.102:0/243526836 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2f8c004300 con 0x7f2f98106ca0 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.998+0000 7f2f977fe700 1 -- 172.21.15.102:0/243526836 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f8c00faf0 con 0x7f2f98106ca0 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.999+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/243526836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 msgr2=0x7f2f981070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.999+0000 7f2f9ef2c700 1 --2- 172.21.15.102:0/243526836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f981070c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0098a0 tx=0x7f2f8c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.999+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/243526836 shutdown_connections 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.999+0000 7f2f9ef2c700 1 --2- 172.21.15.102:0/243526836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f981070c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.999+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/243526836 >> 172.21.15.102:0/243526836 conn(0x7f2f98102220 msgr2=0x7f2f98104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.999+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/243526836 shutdown_connections 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:01.999+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/243526836 wait complete. 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.000+0000 7f2f9ef2c700 1 Processor -- start 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.000+0000 7f2f9ef2c700 1 -- start start 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.000+0000 7f2f9ef2c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f98199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.000+0000 7f2f9ef2c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9807afd0 con 0x7f2f98106ca0 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.000+0000 7f2f9ccc8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f98199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:02.724 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.000+0000 7f2f9ccc8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f98199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54424/0 (socket says 172.21.15.102:54424) 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.000+0000 7f2f9ccc8700 1 -- 172.21.15.102:0/1863385259 learned_addr learned my addr 172.21.15.102:0/1863385259 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.000+0000 7f2f9ccc8700 1 -- 172.21.15.102:0/1863385259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f98199ff0 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.000+0000 7f2f9ccc8700 1 --2- 172.21.15.102:0/1863385259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f98199ab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00f7b0 tx=0x7f2f8c0036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.001+0000 7f2f95ffb700 1 -- 172.21.15.102:0/1863385259 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f8c003b40 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.001+0000 7f2f95ffb700 1 -- 172.21.15.102:0/1863385259 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2f8c010900 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.001+0000 7f2f95ffb700 1 -- 172.21.15.102:0/1863385259 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f8c022920 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.001+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/1863385259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9819a280 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.001+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/1863385259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9819cf70 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.001+0000 7f2f95ffb700 1 -- 172.21.15.102:0/1863385259 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2f8c022bd0 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.001+0000 7f2f95ffb700 1 -- 172.21.15.102:0/1863385259 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2f8c01f070 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.002+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/1863385259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f98063890 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.006+0000 7f2f95ffb700 1 -- 172.21.15.102:0/1863385259 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2f8c015070 con 0x7f2f98106ca0 2024-04-24T23:03:02.725 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.077+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/1863385259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2f9802dda0 con 0x7f2f98106ca0 2024-04-24T23:03:02.726 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.078+0000 7f2f95ffb700 1 -- 172.21.15.102:0/1863385259 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2f8c010ab0 con 0x7f2f98106ca0 2024-04-24T23:03:02.726 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.079+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/1863385259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 msgr2=0x7f2f98199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:02.726 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.079+0000 7f2f9ef2c700 1 --2- 172.21.15.102:0/1863385259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f98199ab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00f7b0 tx=0x7f2f8c0036a0 comp rx=0 tx=0).stop 2024-04-24T23:03:02.726 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.079+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/1863385259 shutdown_connections 2024-04-24T23:03:02.726 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.079+0000 7f2f9ef2c700 1 --2- 172.21.15.102:0/1863385259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f98106ca0 0x7f2f98199ab0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:02.726 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.079+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/1863385259 >> 172.21.15.102:0/1863385259 conn(0x7f2f98102220 msgr2=0x7f2f98190670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:02.726 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.079+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/1863385259 shutdown_connections 2024-04-24T23:03:02.726 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:02.079+0000 7f2f9ef2c700 1 -- 172.21.15.102:0/1863385259 wait complete. 2024-04-24T23:03:02.726 INFO:teuthology.orchestra.run.smithi102.stdout:mgr not available, waiting (1/15)... 2024-04-24T23:03:03.822 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:03 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:03.493+0000 7f1d61028200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:03:03.822 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:03 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:03.661+0000 7f1d61028200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:03:04.117 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:03 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:03.831+0000 7f1d61028200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:03:04.117 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:04 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:04.115+0000 7f1d61028200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:03:04.409 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:04 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:04.205+0000 7f1d61028200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:03:04.409 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:04 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:04.407+0000 7f1d61028200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:03:04.821 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:04 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:04.597+0000 7f1d61028200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:03:05.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:05 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/539779577' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:03:05.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:05.223+0000 7f1d61028200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:03:05.940 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:05.648+0000 7f1d61028200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:03:05.941 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:05.742+0000 7f1d61028200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:03:05.941 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:05.938+0000 7f1d61028200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:03:05.965 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:05.965 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:05.965 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:03:05.966 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:03:05.967 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:03:05.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:03:05.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:03:05.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:03:05.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:05.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:03:05.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:03:05.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:03:05.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:03:05.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:03:05.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:02:53.253856+0000", 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:05.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.171+0000 7fd181044700 1 Processor -- start 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.171+0000 7fd181044700 1 -- start start 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.171+0000 7fd181044700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c10acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.171+0000 7fd181044700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c07be70 con 0x7fd17c1088c0 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.171+0000 7fd17ad9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c10acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.171+0000 7fd17ad9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c10acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54432/0 (socket says 172.21.15.102:54432) 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.171+0000 7fd17ad9d700 1 -- 172.21.15.102:0/2967795312 learned_addr learned my addr 172.21.15.102:0/2967795312 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd17ad9d700 1 -- 172.21.15.102:0/2967795312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c10b1f0 con 0x7fd17c1088c0 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd17ad9d700 1 --2- 172.21.15.102:0/2967795312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c10acb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1700098a0 tx=0x7fd170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9f41ca98b7bcff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd179d9b700 1 -- 172.21.15.102:0/2967795312 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd170004160 con 0x7fd17c1088c0 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd179d9b700 1 -- 172.21.15.102:0/2967795312 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd170004300 con 0x7fd17c1088c0 2024-04-24T23:03:05.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd179d9b700 1 -- 172.21.15.102:0/2967795312 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd170011a80 con 0x7fd17c1088c0 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd181044700 1 -- 172.21.15.102:0/2967795312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 msgr2=0x7fd17c10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd181044700 1 --2- 172.21.15.102:0/2967795312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c10acb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1700098a0 tx=0x7fd170009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd181044700 1 -- 172.21.15.102:0/2967795312 shutdown_connections 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd181044700 1 --2- 172.21.15.102:0/2967795312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c10acb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.172+0000 7fd181044700 1 -- 172.21.15.102:0/2967795312 >> 172.21.15.102:0/2967795312 conn(0x7fd17c1022b0 msgr2=0x7fd17c1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.173+0000 7fd181044700 1 -- 172.21.15.102:0/2967795312 shutdown_connections 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.173+0000 7fd181044700 1 -- 172.21.15.102:0/2967795312 wait complete. 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.173+0000 7fd181044700 1 Processor -- start 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.173+0000 7fd181044700 1 -- start start 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd181044700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd181044700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c10c0d0 con 0x7fd17c1088c0 2024-04-24T23:03:05.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd17ad9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd17ad9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54440/0 (socket says 172.21.15.102:54440) 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd17ad9d700 1 -- 172.21.15.102:0/539779577 learned_addr learned my addr 172.21.15.102:0/539779577 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd17ad9d700 1 -- 172.21.15.102:0/539779577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c19c210 con 0x7fd17c1088c0 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd17ad9d700 1 --2- 172.21.15.102:0/539779577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c19bcd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd170006af0 tx=0x7fd170011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd163fff700 1 -- 172.21.15.102:0/539779577 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd170003930 con 0x7fd17c1088c0 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd163fff700 1 -- 172.21.15.102:0/539779577 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd170003ad0 con 0x7fd17c1088c0 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.174+0000 7fd163fff700 1 -- 172.21.15.102:0/539779577 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1700195e0 con 0x7fd17c1088c0 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.175+0000 7fd181044700 1 -- 172.21.15.102:0/539779577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17c19c4a0 con 0x7fd17c1088c0 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.175+0000 7fd181044700 1 -- 172.21.15.102:0/539779577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17c19f180 con 0x7fd17c1088c0 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.175+0000 7fd163fff700 1 -- 172.21.15.102:0/539779577 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd170019c20 con 0x7fd17c1088c0 2024-04-24T23:03:05.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.175+0000 7fd163fff700 1 -- 172.21.15.102:0/539779577 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd170024730 con 0x7fd17c1088c0 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.176+0000 7fd181044700 1 -- 172.21.15.102:0/539779577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17c063890 con 0x7fd17c1088c0 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.180+0000 7fd163fff700 1 -- 172.21.15.102:0/539779577 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd170043dd0 con 0x7fd17c1088c0 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.266+0000 7fd181044700 1 -- 172.21.15.102:0/539779577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd17c19f350 con 0x7fd17c1088c0 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.267+0000 7fd163fff700 1 -- 172.21.15.102:0/539779577 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd170032030 con 0x7fd17c1088c0 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.268+0000 7fd181044700 1 -- 172.21.15.102:0/539779577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 msgr2=0x7fd17c19bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.268+0000 7fd181044700 1 --2- 172.21.15.102:0/539779577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c19bcd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd170006af0 tx=0x7fd170011b70 comp rx=0 tx=0).stop 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.268+0000 7fd181044700 1 -- 172.21.15.102:0/539779577 shutdown_connections 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.268+0000 7fd181044700 1 --2- 172.21.15.102:0/539779577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd17c1088c0 0x7fd17c19bcd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.268+0000 7fd181044700 1 -- 172.21.15.102:0/539779577 >> 172.21.15.102:0/539779577 conn(0x7fd17c1022b0 msgr2=0x7fd17c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.268+0000 7fd181044700 1 -- 172.21.15.102:0/539779577 shutdown_connections 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:05.268+0000 7fd181044700 1 -- 172.21.15.102:0/539779577 wait complete. 2024-04-24T23:03:05.982 INFO:teuthology.orchestra.run.smithi102.stdout:mgr not available, waiting (2/15)... 2024-04-24T23:03:06.799 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:06 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:06.466+0000 7f1d61028200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:03:06.800 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:06 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:06.555+0000 7f1d61028200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:03:07.074 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:06 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:06.798+0000 7f1d61028200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:03:07.074 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:06 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:06.930+0000 7f1d61028200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:03:07.074 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:07 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:07.025+0000 7f1d61028200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:03:07.571 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:07 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:07.139+0000 7f1d61028200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:03:07.571 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:07 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:07.230+0000 7f1d61028200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:03:08.321 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:08.058+0000 7f1d61028200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:03:08.322 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:08.270+0000 7f1d61028200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:03:08.773 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: Activating manager daemon x 2024-04-24T23:03:08.773 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: mgrmap e2: x(active, starting, since 0.00294138s) 2024-04-24T23:03:08.773 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:03:08.773 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:03:08.774 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:03:08.774 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:03:08.774 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:03:08.774 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: Manager daemon x is now available 2024-04-24T23:03:08.774 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:03:08.774 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:03:08.774 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' 2024-04-24T23:03:08.774 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' 2024-04-24T23:03:08.774 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:08 smithi102 ceph-mon[29216]: from='mgr.14100 172.21.15.102:0/4155045001' entity='mgr.x' 2024-04-24T23:03:09.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:03:09.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:03:09.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:03:09.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:02:53.253856+0000", 2024-04-24T23:03:09.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.393+0000 7f733b4a3700 1 Processor -- start 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.393+0000 7f733b4a3700 1 -- start start 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.393+0000 7f733b4a3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f73341081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.393+0000 7f733b4a3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7334108770 con 0x7f7334107d80 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.393+0000 7f733923f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f73341081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.393+0000 7f733923f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f73341081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37276/0 (socket says 172.21.15.102:37276) 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.393+0000 7f733923f700 1 -- 172.21.15.102:0/1589749856 learned_addr learned my addr 172.21.15.102:0/1589749856 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.393+0000 7f733923f700 1 -- 172.21.15.102:0/1589749856 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7334108f80 con 0x7f7334107d80 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f733923f700 1 --2- 172.21.15.102:0/1589749856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f73341081a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f73300098a0 tx=0x7f7330009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd9e7e696ab61d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f732bfff700 1 -- 172.21.15.102:0/1589749856 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7330004160 con 0x7f7334107d80 2024-04-24T23:03:09.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f732bfff700 1 -- 172.21.15.102:0/1589749856 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7330004300 con 0x7f7334107d80 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f732bfff700 1 -- 172.21.15.102:0/1589749856 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7330011a80 con 0x7f7334107d80 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f733b4a3700 1 -- 172.21.15.102:0/1589749856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 msgr2=0x7f73341081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f733b4a3700 1 --2- 172.21.15.102:0/1589749856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f73341081a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f73300098a0 tx=0x7f7330009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f733b4a3700 1 -- 172.21.15.102:0/1589749856 shutdown_connections 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f733b4a3700 1 --2- 172.21.15.102:0/1589749856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f73341081a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f733b4a3700 1 -- 172.21.15.102:0/1589749856 >> 172.21.15.102:0/1589749856 conn(0x7f7334103320 msgr2=0x7f7334105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.394+0000 7f733b4a3700 1 -- 172.21.15.102:0/1589749856 shutdown_connections 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.395+0000 7f733b4a3700 1 -- 172.21.15.102:0/1589749856 wait complete. 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.395+0000 7f733b4a3700 1 Processor -- start 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.395+0000 7f733b4a3700 1 -- start start 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.395+0000 7f733b4a3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f733407db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.395+0000 7f733b4a3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f733410c1b0 con 0x7f7334107d80 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.395+0000 7f733923f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f733407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.395+0000 7f733923f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f733407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37278/0 (socket says 172.21.15.102:37278) 2024-04-24T23:03:09.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.395+0000 7f733923f700 1 -- 172.21.15.102:0/1232469145 learned_addr learned my addr 172.21.15.102:0/1232469145 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.396+0000 7f733923f700 1 -- 172.21.15.102:0/1232469145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733407e0b0 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.396+0000 7f733923f700 1 --2- 172.21.15.102:0/1232469145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f733407db70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7330006af0 tx=0x7f7330011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.396+0000 7f732a7fc700 1 -- 172.21.15.102:0/1232469145 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7330003930 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.396+0000 7f732a7fc700 1 -- 172.21.15.102:0/1232469145 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7330003ad0 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.396+0000 7f732a7fc700 1 -- 172.21.15.102:0/1232469145 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73300195e0 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.396+0000 7f733b4a3700 1 -- 172.21.15.102:0/1232469145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733407c2a0 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.396+0000 7f733b4a3700 1 -- 172.21.15.102:0/1232469145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733407c7e0 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.397+0000 7f732a7fc700 1 -- 172.21.15.102:0/1232469145 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 2) ==== 46043+0+0 (secure 0 0 0) 0x7f73300198e0 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.397+0000 7f733b4a3700 1 -- 172.21.15.102:0/1232469145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7334063890 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.397+0000 7f732a7fc700 1 -- 172.21.15.102:0/1232469145 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f733004d030 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.399+0000 7f732a7fc700 1 -- 172.21.15.102:0/1232469145 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f733000fda0 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.470+0000 7f733b4a3700 1 -- 172.21.15.102:0/1232469145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f733407ca60 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.470+0000 7f732a7fc700 1 -- 172.21.15.102:0/1232469145 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f733001a4b0 con 0x7f7334107d80 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.471+0000 7f733b4a3700 1 -- 172.21.15.102:0/1232469145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 msgr2=0x7f733407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:09.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.471+0000 7f733b4a3700 1 --2- 172.21.15.102:0/1232469145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f733407db70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7330006af0 tx=0x7f7330011b70 comp rx=0 tx=0).stop 2024-04-24T23:03:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.471+0000 7f733b4a3700 1 -- 172.21.15.102:0/1232469145 shutdown_connections 2024-04-24T23:03:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.471+0000 7f733b4a3700 1 --2- 172.21.15.102:0/1232469145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7334107d80 0x7f733407db70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.471+0000 7f733b4a3700 1 -- 172.21.15.102:0/1232469145 >> 172.21.15.102:0/1232469145 conn(0x7f7334103320 msgr2=0x7f7334103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.471+0000 7f733b4a3700 1 -- 172.21.15.102:0/1232469145 shutdown_connections 2024-04-24T23:03:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:08.471+0000 7f733b4a3700 1 -- 172.21.15.102:0/1232469145 wait complete. 2024-04-24T23:03:09.086 INFO:teuthology.orchestra.run.smithi102.stdout:mgr not available, waiting (3/15)... 2024-04-24T23:03:09.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:09 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1232469145' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:03:09.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:09 smithi102 ceph-mon[29216]: mgrmap e3: x(active, since 1.00655s) 2024-04-24T23:03:11.452 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:11 smithi102 ceph-mon[29216]: mgrmap e4: x(active, since 2s) 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:03:12.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:03:12.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:03:12.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:03:12.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:02:53.253856+0000", 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.489+0000 7f113a85b700 1 Processor -- start 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.489+0000 7f113a85b700 1 -- start start 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.490+0000 7f113a85b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f11341081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:12.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.490+0000 7f113a85b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1134108770 con 0x7f1134107d80 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.490+0000 7f1133fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f11341081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.490+0000 7f1133fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f11341081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37292/0 (socket says 172.21.15.102:37292) 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.490+0000 7f1133fff700 1 -- 172.21.15.102:0/2604027856 learned_addr learned my addr 172.21.15.102:0/2604027856 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.490+0000 7f1133fff700 1 -- 172.21.15.102:0/2604027856 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1134108f80 con 0x7f1134107d80 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.490+0000 7f1133fff700 1 --2- 172.21.15.102:0/2604027856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f11341081a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f11240098a0 tx=0x7f1124009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb593aa850d3022a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.490+0000 7f11337fe700 1 -- 172.21.15.102:0/2604027856 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1124004160 con 0x7f1134107d80 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.491+0000 7f11337fe700 1 -- 172.21.15.102:0/2604027856 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1124004300 con 0x7f1134107d80 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.491+0000 7f11337fe700 1 -- 172.21.15.102:0/2604027856 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1124011a80 con 0x7f1134107d80 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.491+0000 7f113a85b700 1 -- 172.21.15.102:0/2604027856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 msgr2=0x7f11341081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.491+0000 7f113a85b700 1 --2- 172.21.15.102:0/2604027856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f11341081a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f11240098a0 tx=0x7f1124009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.491+0000 7f113a85b700 1 -- 172.21.15.102:0/2604027856 shutdown_connections 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.491+0000 7f113a85b700 1 --2- 172.21.15.102:0/2604027856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f11341081a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.491+0000 7f113a85b700 1 -- 172.21.15.102:0/2604027856 >> 172.21.15.102:0/2604027856 conn(0x7f1134103320 msgr2=0x7f1134105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:12.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.491+0000 7f113a85b700 1 -- 172.21.15.102:0/2604027856 shutdown_connections 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.491+0000 7f113a85b700 1 -- 172.21.15.102:0/2604027856 wait complete. 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.492+0000 7f113a85b700 1 Processor -- start 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.492+0000 7f113a85b700 1 -- start start 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.492+0000 7f113a85b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f113407db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.492+0000 7f113a85b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f113410c1b0 con 0x7f1134107d80 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.492+0000 7f1133fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f113407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.492+0000 7f1133fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f113407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37302/0 (socket says 172.21.15.102:37302) 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.492+0000 7f1133fff700 1 -- 172.21.15.102:0/4017739235 learned_addr learned my addr 172.21.15.102:0/4017739235 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.492+0000 7f1133fff700 1 -- 172.21.15.102:0/4017739235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113407e0b0 con 0x7f1134107d80 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.492+0000 7f1133fff700 1 --2- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f113407db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1124006af0 tx=0x7f1124011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.493+0000 7f1131ffb700 1 -- 172.21.15.102:0/4017739235 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1124003930 con 0x7f1134107d80 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.493+0000 7f1131ffb700 1 -- 172.21.15.102:0/4017739235 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1124003ad0 con 0x7f1134107d80 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.493+0000 7f1131ffb700 1 -- 172.21.15.102:0/4017739235 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11240195e0 con 0x7f1134107d80 2024-04-24T23:03:12.325 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.493+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113407c2a0 con 0x7f1134107d80 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.493+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113407c7e0 con 0x7f1134107d80 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.494+0000 7f1131ffb700 1 -- 172.21.15.102:0/4017739235 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f1124019890 con 0x7f1134107d80 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.494+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113410c330 con 0x7f1134107d80 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.494+0000 7f1131ffb700 1 --2- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f1120039000 0x7f112003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.494+0000 7f1131ffb700 1 -- 172.21.15.102:0/4017739235 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f112404d630 con 0x7f1134107d80 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.494+0000 7f112bfff700 1 --2- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f1120039000 0x7f112003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.494+0000 7f112bfff700 1 --2- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f1120039000 0x7f112003b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f111c006fd0 tx=0x7f111c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.499+0000 7f1131ffb700 1 -- 172.21.15.102:0/4017739235 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f112404d050 con 0x7f1134107d80 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.772+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f113407cb20 con 0x7f1134107d80 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.773+0000 7f1131ffb700 1 -- 172.21.15.102:0/4017739235 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f113407cb20 con 0x7f1134107d80 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f1120039000 msgr2=0x7f112003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 --2- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f1120039000 0x7f112003b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f111c006fd0 tx=0x7f111c006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 msgr2=0x7f113407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:12.326 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 --2- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f113407db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1124006af0 tx=0x7f1124011b70 comp rx=0 tx=0).stop 2024-04-24T23:03:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 shutdown_connections 2024-04-24T23:03:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 --2- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f1120039000 0x7f112003b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 --2- 172.21.15.102:0/4017739235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1134107d80 0x7f113407db70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 >> 172.21.15.102:0/4017739235 conn(0x7f1134103320 msgr2=0x7f1134103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 shutdown_connections 2024-04-24T23:03:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:11.775+0000 7f113a85b700 1 -- 172.21.15.102:0/4017739235 wait complete. 2024-04-24T23:03:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:mgr is available 2024-04-24T23:03:12.431 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:12 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4017739235' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:03:13.572 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout fsid = 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:03:13.573 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.755+0000 7fdc39ef8700 1 Processor -- start 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.755+0000 7fdc39ef8700 1 -- start start 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.756+0000 7fdc39ef8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.756+0000 7fdc39ef8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3407c8d0 con 0x7fdc3407dea0 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.756+0000 7fdc337fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.756+0000 7fdc337fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37310/0 (socket says 172.21.15.102:37310) 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.756+0000 7fdc337fe700 1 -- 172.21.15.102:0/3502043178 learned_addr learned my addr 172.21.15.102:0/3502043178 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.756+0000 7fdc337fe700 1 -- 172.21.15.102:0/3502043178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3407ca50 con 0x7fdc3407dea0 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.756+0000 7fdc337fe700 1 --2- 172.21.15.102:0/3502043178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3407c300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098a0 tx=0x7fdc24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b035825e513fef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.756+0000 7fdc327fc700 1 -- 172.21.15.102:0/3502043178 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc24004160 con 0x7fdc3407dea0 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.756+0000 7fdc327fc700 1 -- 172.21.15.102:0/3502043178 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdc24004300 con 0x7fdc3407dea0 2024-04-24T23:03:13.574 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.757+0000 7fdc327fc700 1 -- 172.21.15.102:0/3502043178 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc24011a80 con 0x7fdc3407dea0 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.757+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3502043178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 msgr2=0x7fdc3407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.757+0000 7fdc39ef8700 1 --2- 172.21.15.102:0/3502043178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3407c300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098a0 tx=0x7fdc24009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.757+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3502043178 shutdown_connections 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.757+0000 7fdc39ef8700 1 --2- 172.21.15.102:0/3502043178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3407c300 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.757+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3502043178 >> 172.21.15.102:0/3502043178 conn(0x7fdc341032a0 msgr2=0x7fdc34105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.757+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3502043178 shutdown_connections 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.757+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3502043178 wait complete. 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.758+0000 7fdc39ef8700 1 Processor -- start 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.758+0000 7fdc39ef8700 1 -- start start 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.758+0000 7fdc39ef8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.758+0000 7fdc39ef8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3410e290 con 0x7fdc3407dea0 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.758+0000 7fdc337fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:13.575 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.758+0000 7fdc337fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3419df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37324/0 (socket says 172.21.15.102:37324) 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.758+0000 7fdc337fe700 1 -- 172.21.15.102:0/3407610163 learned_addr learned my addr 172.21.15.102:0/3407610163 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.758+0000 7fdc337fe700 1 -- 172.21.15.102:0/3407610163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3419e450 con 0x7fdc3407dea0 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.758+0000 7fdc337fe700 1 --2- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3419df10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc24006af0 tx=0x7fdc24011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.759+0000 7fdc30ff9700 1 -- 172.21.15.102:0/3407610163 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc24003930 con 0x7fdc3407dea0 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.759+0000 7fdc30ff9700 1 -- 172.21.15.102:0/3407610163 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdc24003ad0 con 0x7fdc3407dea0 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.759+0000 7fdc30ff9700 1 -- 172.21.15.102:0/3407610163 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc240195e0 con 0x7fdc3407dea0 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.759+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3419e6e0 con 0x7fdc3407dea0 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.759+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc341a13d0 con 0x7fdc3407dea0 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.760+0000 7fdc30ff9700 1 -- 172.21.15.102:0/3407610163 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fdc24019890 con 0x7fdc3407dea0 2024-04-24T23:03:13.576 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.760+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc34063890 con 0x7fdc3407dea0 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.760+0000 7fdc30ff9700 1 --2- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fdc1c039000 0x7fdc1c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.760+0000 7fdc30ff9700 1 -- 172.21.15.102:0/3407610163 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdc2404d730 con 0x7fdc3407dea0 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.760+0000 7fdc32ffd700 1 --2- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fdc1c039000 0x7fdc1c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.760+0000 7fdc32ffd700 1 --2- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fdc1c039000 0x7fdc1c03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc28006fd0 tx=0x7fdc28006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.765+0000 7fdc30ff9700 1 -- 172.21.15.102:0/3407610163 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdc2404d150 con 0x7fdc3407dea0 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.957+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fdc341a15a0 con 0x7fdc3407dea0 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.958+0000 7fdc30ff9700 1 -- 172.21.15.102:0/3407610163 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+398 (secure 0 0 0) 0x7fdc341a15a0 con 0x7fdc3407dea0 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fdc1c039000 msgr2=0x7fdc1c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 --2- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fdc1c039000 0x7fdc1c03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc28006fd0 tx=0x7fdc28006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 msgr2=0x7fdc3419df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 --2- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3419df10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc24006af0 tx=0x7fdc24011b70 comp rx=0 tx=0).stop 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 shutdown_connections 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 --2- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fdc1c039000 0x7fdc1c03b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:13.577 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 --2- 172.21.15.102:0/3407610163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc3407dea0 0x7fdc3419df10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 >> 172.21.15.102:0/3407610163 conn(0x7fdc341032a0 msgr2=0x7fdc34103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 shutdown_connections 2024-04-24T23:03:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:12.960+0000 7fdc39ef8700 1 -- 172.21.15.102:0/3407610163 wait complete. 2024-04-24T23:03:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling cephadm module... 2024-04-24T23:03:13.691 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:13 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3407610163' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:03:14.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:14 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1322308871' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T23:03:14.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:14 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: ignoring --setuser ceph since I am not root 2024-04-24T23:03:14.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:14 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: ignoring --setgroup ceph since I am not root 2024-04-24T23:03:14.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:14 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:14.494+0000 7f205932e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:03:14.960 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:14 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:14.681+0000 7f205932e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:03:14.961 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:14 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:14.784+0000 7f205932e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:03:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.022+0000 7f075d108700 1 Processor -- start 2024-04-24T23:03:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.022+0000 7f075d108700 1 -- start start 2024-04-24T23:03:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.022+0000 7f075d108700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075810ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.022+0000 7f075d108700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f075807be70 con 0x7f0758108890 2024-04-24T23:03:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.022+0000 7f0756d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075810ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.022+0000 7f0756d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075810ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37330/0 (socket says 172.21.15.102:37330) 2024-04-24T23:03:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.022+0000 7f0756d9d700 1 -- 172.21.15.102:0/3354827796 learned_addr learned my addr 172.21.15.102:0/3354827796 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:14.968 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.022+0000 7f0756d9d700 1 -- 172.21.15.102:0/3354827796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075810b1c0 con 0x7f0758108890 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f0756d9d700 1 --2- 172.21.15.102:0/3354827796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075810ac80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f07480098a0 tx=0x7f0748009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e0af396e9f7553 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f0755d9b700 1 -- 172.21.15.102:0/3354827796 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0748004160 con 0x7f0758108890 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f0755d9b700 1 -- 172.21.15.102:0/3354827796 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0748004300 con 0x7f0758108890 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f0755d9b700 1 -- 172.21.15.102:0/3354827796 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0748011a80 con 0x7f0758108890 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f075d108700 1 -- 172.21.15.102:0/3354827796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 msgr2=0x7f075810ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f075d108700 1 --2- 172.21.15.102:0/3354827796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075810ac80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f07480098a0 tx=0x7f0748009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f075d108700 1 -- 172.21.15.102:0/3354827796 shutdown_connections 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f075d108700 1 --2- 172.21.15.102:0/3354827796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075810ac80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f075d108700 1 -- 172.21.15.102:0/3354827796 >> 172.21.15.102:0/3354827796 conn(0x7f0758102280 msgr2=0x7f07581046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f075d108700 1 -- 172.21.15.102:0/3354827796 shutdown_connections 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.023+0000 7f075d108700 1 -- 172.21.15.102:0/3354827796 wait complete. 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.024+0000 7f075d108700 1 Processor -- start 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.024+0000 7f075d108700 1 -- start start 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.024+0000 7f075d108700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:14.969 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.024+0000 7f075d108700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f075810c110 con 0x7f0758108890 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f0756d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f0756d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075819bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37338/0 (socket says 172.21.15.102:37338) 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f0756d9d700 1 -- 172.21.15.102:0/1322308871 learned_addr learned my addr 172.21.15.102:0/1322308871 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f0756d9d700 1 -- 172.21.15.102:0/1322308871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075819c200 con 0x7f0758108890 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f0756d9d700 1 --2- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075819bcc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0748006af0 tx=0x7f0748011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f073ffff700 1 -- 172.21.15.102:0/1322308871 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0748003930 con 0x7f0758108890 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f073ffff700 1 -- 172.21.15.102:0/1322308871 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0748003ad0 con 0x7f0758108890 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075819c490 con 0x7f0758108890 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f073ffff700 1 -- 172.21.15.102:0/1322308871 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07480195e0 con 0x7f0758108890 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.025+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075819f180 con 0x7f0758108890 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.026+0000 7f073ffff700 1 -- 172.21.15.102:0/1322308871 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f07480198e0 con 0x7f0758108890 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.026+0000 7f073ffff700 1 --2- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0740038fb0 0x7f074003b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:14.970 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.026+0000 7f073ffff700 1 -- 172.21.15.102:0/1322308871 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f074804d680 con 0x7f0758108890 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.026+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0758063890 con 0x7f0758108890 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.026+0000 7f075659c700 1 --2- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0740038fb0 0x7f074003b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.027+0000 7f075659c700 1 --2- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0740038fb0 0x7f074003b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f074c006fd0 tx=0x7f074c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.031+0000 7f073ffff700 1 -- 172.21.15.102:0/1322308871 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0748023c90 con 0x7f0758108890 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.283+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f075819f3f0 con 0x7f0758108890 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.330+0000 7f073ffff700 1 -- 172.21.15.102:0/1322308871 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f0748032020 con 0x7f0758108890 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.332+0000 7f073ffff700 1 -- 172.21.15.102:0/1322308871 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f074802d960 con 0x7f0758108890 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0740038fb0 msgr2=0x7f074003b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 --2- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0740038fb0 0x7f074003b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f074c006fd0 tx=0x7f074c006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 msgr2=0x7f075819bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 --2- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075819bcc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0748006af0 tx=0x7f0748011b70 comp rx=0 tx=0).stop 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 shutdown_connections 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 --2- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0740038fb0 0x7f074003b470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:14.971 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 --2- 172.21.15.102:0/1322308871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0758108890 0x7f075819bcc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 >> 172.21.15.102:0/1322308871 conn(0x7f0758102280 msgr2=0x7f0758102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 shutdown_connections 2024-04-24T23:03:14.972 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:14.333+0000 7f075d108700 1 -- 172.21.15.102:0/1322308871 wait complete. 2024-04-24T23:03:15.238 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:15 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:15.100+0000 7f205932e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:03:15.238 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:15 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:15.187+0000 7f205932e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:03:15.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:15 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1322308871' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T23:03:15.543 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:15 smithi102 ceph-mon[29216]: mgrmap e5: x(active, since 6s) 2024-04-24T23:03:16.199 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.335+0000 7fae5c8e0700 1 Processor -- start 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.335+0000 7fae5c8e0700 1 -- start start 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5c8e0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5c8e0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae5807c8f0 con 0x7fae5807dec0 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37364/0 (socket says 172.21.15.102:37364) 2024-04-24T23:03:16.200 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5659c700 1 -- 172.21.15.102:0/3642671187 learned_addr learned my addr 172.21.15.102:0/3642671187 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5659c700 1 -- 172.21.15.102:0/3642671187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5807ca70 con 0x7fae5807dec0 2024-04-24T23:03:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5659c700 1 --2- 172.21.15.102:0/3642671187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5807c320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae48009ab0 tx=0x7fae48009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36f10f053c90e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5559a700 1 -- 172.21.15.102:0/3642671187 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae48004160 con 0x7fae5807dec0 2024-04-24T23:03:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5559a700 1 -- 172.21.15.102:0/3642671187 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fae4800bc70 con 0x7fae5807dec0 2024-04-24T23:03:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.336+0000 7fae5559a700 1 -- 172.21.15.102:0/3642671187 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae48003c30 con 0x7fae5807dec0 2024-04-24T23:03:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.337+0000 7fae5c8e0700 1 -- 172.21.15.102:0/3642671187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 msgr2=0x7fae5807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.337+0000 7fae5c8e0700 1 --2- 172.21.15.102:0/3642671187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5807c320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae48009ab0 tx=0x7fae48009dc0 comp rx=0 tx=0).stop 2024-04-24T23:03:16.201 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.337+0000 7fae5c8e0700 1 -- 172.21.15.102:0/3642671187 shutdown_connections 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.337+0000 7fae5c8e0700 1 --2- 172.21.15.102:0/3642671187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5807c320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.337+0000 7fae5c8e0700 1 -- 172.21.15.102:0/3642671187 >> 172.21.15.102:0/3642671187 conn(0x7fae58103300 msgr2=0x7fae58105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.337+0000 7fae5c8e0700 1 -- 172.21.15.102:0/3642671187 shutdown_connections 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.337+0000 7fae5c8e0700 1 -- 172.21.15.102:0/3642671187 wait complete. 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.338+0000 7fae5c8e0700 1 Processor -- start 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.338+0000 7fae5c8e0700 1 -- start start 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.338+0000 7fae5c8e0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.338+0000 7fae5c8e0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae5810e2b0 con 0x7fae5807dec0 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae5659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:16.202 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae5659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37380/0 (socket says 172.21.15.102:37380) 2024-04-24T23:03:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae5659c700 1 -- 172.21.15.102:0/1271397486 learned_addr learned my addr 172.21.15.102:0/1271397486 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae5659c700 1 -- 172.21.15.102:0/1271397486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5819e410 con 0x7fae5807dec0 2024-04-24T23:03:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae5659c700 1 --2- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5819ded0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae480039c0 tx=0x7fae4800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae477fe700 1 -- 172.21.15.102:0/1271397486 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae48004220 con 0x7fae5807dec0 2024-04-24T23:03:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae477fe700 1 -- 172.21.15.102:0/1271397486 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fae480043c0 con 0x7fae5807dec0 2024-04-24T23:03:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5819e6a0 con 0x7fae5807dec0 2024-04-24T23:03:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae477fe700 1 -- 172.21.15.102:0/1271397486 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae480115b0 con 0x7fae5807dec0 2024-04-24T23:03:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.339+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae581a1390 con 0x7fae5807dec0 2024-04-24T23:03:16.203 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.340+0000 7fae477fe700 1 -- 172.21.15.102:0/1271397486 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fae48011820 con 0x7fae5807dec0 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.340+0000 7fae477fe700 1 --2- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fae40039050 0x7fae4003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.340+0000 7fae477fe700 1 -- 172.21.15.102:0/1271397486 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fae4804e200 con 0x7fae5807dec0 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.340+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae58063890 con 0x7fae5807dec0 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.340+0000 7fae55d9b700 1 -- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fae40039050 msgr2=0x7fae4003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/2363268578 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.340+0000 7fae55d9b700 1 --2- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fae40039050 0x7fae4003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.346+0000 7fae477fe700 1 -- 172.21.15.102:0/1271397486 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fae4801f030 con 0x7fae5807dec0 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.541+0000 7fae55d9b700 1 -- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fae40039050 msgr2=0x7fae4003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/2363268578 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.541+0000 7fae55d9b700 1 --2- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fae40039050 0x7fae4003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.636+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fae581a1870 con 0x7fae5807dec0 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.637+0000 7fae477fe700 1 -- 172.21.15.102:0/1271397486 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fae4802b350 con 0x7fae5807dec0 2024-04-24T23:03:16.204 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fae40039050 msgr2=0x7fae4003b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 --2- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fae40039050 0x7fae4003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 msgr2=0x7fae5819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 --2- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5819ded0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae480039c0 tx=0x7fae4800b890 comp rx=0 tx=0).stop 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 shutdown_connections 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 --2- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7fae40039050 0x7fae4003b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 --2- 172.21.15.102:0/1271397486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae5807dec0 0x7fae5819ded0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 >> 172.21.15.102:0/1271397486 conn(0x7fae58103300 msgr2=0x7fae58103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 shutdown_connections 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:15.639+0000 7fae5c8e0700 1 -- 172.21.15.102:0/1271397486 wait complete. 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for the mgr to restart... 2024-04-24T23:03:16.205 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr epoch 5... 2024-04-24T23:03:16.528 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:16 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1271397486' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:03:16.528 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:16 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:16.390+0000 7f205932e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:03:16.821 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:16 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:16.563+0000 7f205932e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:03:16.822 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:16 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:16.733+0000 7f205932e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:03:17.107 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:17 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:17.014+0000 7f205932e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:03:17.107 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:17 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:17.105+0000 7f205932e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:03:17.494 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:17 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:17.304+0000 7f205932e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:03:17.821 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:17 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:17.492+0000 7f205932e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:03:18.322 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:18 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:18.077+0000 7f205932e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:03:18.779 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:18 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:18.488+0000 7f205932e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:03:18.780 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:18 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:18.581+0000 7f205932e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:03:18.780 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:18 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:18.776+0000 7f205932e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:03:19.571 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:19 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:19.288+0000 7f205932e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:03:19.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:19 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:19.377+0000 7f205932e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:03:19.949 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:19 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:19.612+0000 7f205932e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:03:19.949 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:19 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:19.742+0000 7f205932e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:03:19.949 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:19 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:19.835+0000 7f205932e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:03:19.949 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:19 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:19.947+0000 7f205932e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:03:20.321 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:20 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:20.036+0000 7f205932e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:03:21.092 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:20 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:20.829+0000 7f205932e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:03:21.092 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:21.038+0000 7f205932e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: Active manager daemon x restarted 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: Activating manager daemon x 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: mgrmap e6: x(active, starting, since 0.00333428s) 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: Manager daemon x is now available 2024-04-24T23:03:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:21.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:21.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:21.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:21.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:03:21.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:21 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:03:22.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:22 smithi102 ceph-mon[29216]: Found migration_current of "None". Setting to last migration. 2024-04-24T23:03:22.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:22 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:22.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:22 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:22.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:22 smithi102 ceph-mon[29216]: mgrmap e7: x(active, since 1.00785s) 2024-04-24T23:03:22.633 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:22.633 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T23:03:22.633 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:03:22.633 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:22.633 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.628+0000 7f0e51e48700 1 Processor -- start 2024-04-24T23:03:22.633 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.628+0000 7f0e51e48700 1 -- start start 2024-04-24T23:03:22.633 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.628+0000 7f0e51e48700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c07c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.628+0000 7f0e51e48700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c07c910 con 0x7f0e4c07dee0 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.628+0000 7f0e4b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c07c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.628+0000 7f0e4b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c07c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40728/0 (socket says 172.21.15.102:40728) 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.628+0000 7f0e4b7fe700 1 -- 172.21.15.102:0/1196312411 learned_addr learned my addr 172.21.15.102:0/1196312411 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.629+0000 7f0e4b7fe700 1 -- 172.21.15.102:0/1196312411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4c07ca90 con 0x7f0e4c07dee0 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.629+0000 7f0e4b7fe700 1 --2- 172.21.15.102:0/1196312411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c07c340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0098a0 tx=0x7f0e3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655440a467704b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.629+0000 7f0e4a7fc700 1 -- 172.21.15.102:0/1196312411 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e3c004160 con 0x7f0e4c07dee0 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.629+0000 7f0e4a7fc700 1 -- 172.21.15.102:0/1196312411 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0e3c004300 con 0x7f0e4c07dee0 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.629+0000 7f0e4a7fc700 1 -- 172.21.15.102:0/1196312411 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e3c011a80 con 0x7f0e4c07dee0 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.630+0000 7f0e51e48700 1 -- 172.21.15.102:0/1196312411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 msgr2=0x7f0e4c07c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.630+0000 7f0e51e48700 1 --2- 172.21.15.102:0/1196312411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c07c340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0098a0 tx=0x7f0e3c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.630+0000 7f0e51e48700 1 -- 172.21.15.102:0/1196312411 shutdown_connections 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.630+0000 7f0e51e48700 1 --2- 172.21.15.102:0/1196312411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c07c340 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.630+0000 7f0e51e48700 1 -- 172.21.15.102:0/1196312411 >> 172.21.15.102:0/1196312411 conn(0x7f0e4c103380 msgr2=0x7f0e4c1057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.630+0000 7f0e51e48700 1 -- 172.21.15.102:0/1196312411 shutdown_connections 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.630+0000 7f0e51e48700 1 -- 172.21.15.102:0/1196312411 wait complete. 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.630+0000 7f0e51e48700 1 Processor -- start 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.631+0000 7f0e51e48700 1 -- start start 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.631+0000 7f0e51e48700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.631+0000 7f0e51e48700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c10e2f0 con 0x7f0e4c07dee0 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.631+0000 7f0e4b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.631+0000 7f0e4b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c19df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40740/0 (socket says 172.21.15.102:40740) 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.631+0000 7f0e4b7fe700 1 -- 172.21.15.102:0/1010036212 learned_addr learned my addr 172.21.15.102:0/1010036212 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.631+0000 7f0e4b7fe700 1 -- 172.21.15.102:0/1010036212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4c19e460 con 0x7f0e4c07dee0 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.631+0000 7f0e4b7fe700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c19df20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c011740 tx=0x7f0e3c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.631+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e3c003af0 con 0x7f0e4c07dee0 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.632+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4c19e6f0 con 0x7f0e4c07dee0 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.632+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0e3c003c90 con 0x7f0e4c07dee0 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.632+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e3c0247d0 con 0x7f0e4c07dee0 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.632+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4c1a13e0 con 0x7f0e4c07dee0 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.632+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f0e3c022020 con 0x7f0e4c07dee0 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.633+0000 7f0e48ff9700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 0x7f0e3403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.633+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 --> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0e4c063890 con 0x7f0e34039000 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.633+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e3c04d0c0 con 0x7f0e4c07dee0 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.633+0000 7f0e4affd700 1 -- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 msgr2=0x7f0e3403b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/2363268578 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.633+0000 7f0e4affd700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 0x7f0e3403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.834+0000 7f0e4affd700 1 -- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 msgr2=0x7f0e3403b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/2363268578 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:16.834+0000 7f0e4affd700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 0x7f0e3403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:03:22.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:17.235+0000 7f0e4affd700 1 -- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 msgr2=0x7f0e3403b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/2363268578 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:17.235+0000 7f0e4affd700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 0x7f0e3403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:18.036+0000 7f0e4affd700 1 -- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 msgr2=0x7f0e3403b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/2363268578 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:18.036+0000 7f0e4affd700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 0x7f0e3403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:19.637+0000 7f0e4affd700 1 -- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 msgr2=0x7f0e3403b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/2363268578 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:19.637+0000 7f0e4affd700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 0x7f0e3403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:21.041+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f0e3c02d770 con 0x7f0e4c07dee0 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:21.041+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 msgr2=0x7f0e3403b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:21.041+0000 7f0e48ff9700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 0x7f0e3403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.045+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f0e3c04ee30 con 0x7f0e4c07dee0 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.045+0000 7f0e48ff9700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e3403c740 0x7f0e3403eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.045+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0e3c021e70 con 0x7f0e3403c740 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.045+0000 7f0e4affd700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e3403c740 0x7f0e3403eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.046+0000 7f0e4affd700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e3403c740 0x7f0e3403eb30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0e40003a10 tx=0x7f0e40009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:22.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.046+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 <== mgr.14120 v2:172.21.15.102:6800/3017038970 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0e3c021e70 con 0x7f0e3403c740 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0e4c1046b0 con 0x7f0e3403c740 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e48ff9700 1 -- 172.21.15.102:0/1010036212 <== mgr.14120 v2:172.21.15.102:6800/3017038970 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f0e4c1046b0 con 0x7f0e3403c740 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e3403c740 msgr2=0x7f0e3403eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e3403c740 0x7f0e3403eb30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0e40003a10 tx=0x7f0e40009290 comp rx=0 tx=0).stop 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 msgr2=0x7f0e4c19df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c19df20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c011740 tx=0x7f0e3c011b00 comp rx=0 tx=0).stop 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 shutdown_connections 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e3403c740 0x7f0e3403eb30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e4c07dee0 0x7f0e4c19df20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 --2- 172.21.15.102:0/1010036212 >> [v2:172.21.15.102:6800/2363268578,v1:172.21.15.102:6801/2363268578] conn(0x7f0e34039000 0x7f0e3403b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.053+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 >> 172.21.15.102:0/1010036212 conn(0x7f0e4c103380 msgr2=0x7f0e4c103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.054+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 shutdown_connections 2024-04-24T23:03:22.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:22.054+0000 7f0e51e48700 1 -- 172.21.15.102:0/1010036212 wait complete. 2024-04-24T23:03:22.639 INFO:teuthology.orchestra.run.smithi102.stdout:mgr epoch 5 is available 2024-04-24T23:03:22.639 INFO:teuthology.orchestra.run.smithi102.stdout:Setting orchestrator backend to cephadm... 2024-04-24T23:03:23.279 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:23 smithi102 ceph-mon[29216]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:03:23.279 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:23 smithi102 ceph-mon[29216]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:03:23.830 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.048+0000 7f7d4ea26700 1 Processor -- start 2024-04-24T23:03:23.830 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.048+0000 7f7d4ea26700 1 -- start start 2024-04-24T23:03:23.830 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.048+0000 7f7d4ea26700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d4810ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:23.830 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.048+0000 7f7d4ea26700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4807bdd0 con 0x7f7d481088a0 2024-04-24T23:03:23.830 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.048+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d4810ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:23.830 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.048+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d4810ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40810/0 (socket says 172.21.15.102:40810) 2024-04-24T23:03:23.830 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.048+0000 7f7d47fff700 1 -- 172.21.15.102:0/423765377 learned_addr learned my addr 172.21.15.102:0/423765377 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:23.830 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.049+0000 7f7d47fff700 1 -- 172.21.15.102:0/423765377 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4810b1d0 con 0x7f7d481088a0 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.049+0000 7f7d47fff700 1 --2- 172.21.15.102:0/423765377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d4810ac90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0098a0 tx=0x7f7d3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45d281a4e76f3f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.049+0000 7f7d46ffd700 1 -- 172.21.15.102:0/423765377 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d3c004160 con 0x7f7d481088a0 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.049+0000 7f7d46ffd700 1 -- 172.21.15.102:0/423765377 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7d3c004300 con 0x7f7d481088a0 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.049+0000 7f7d46ffd700 1 -- 172.21.15.102:0/423765377 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d3c011a80 con 0x7f7d481088a0 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.049+0000 7f7d4ea26700 1 -- 172.21.15.102:0/423765377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 msgr2=0x7f7d4810ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.050+0000 7f7d4ea26700 1 --2- 172.21.15.102:0/423765377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d4810ac90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0098a0 tx=0x7f7d3c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.050+0000 7f7d4ea26700 1 -- 172.21.15.102:0/423765377 shutdown_connections 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.050+0000 7f7d4ea26700 1 --2- 172.21.15.102:0/423765377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d4810ac90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.050+0000 7f7d4ea26700 1 -- 172.21.15.102:0/423765377 >> 172.21.15.102:0/423765377 conn(0x7f7d481021d0 msgr2=0x7f7d48104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.050+0000 7f7d4ea26700 1 -- 172.21.15.102:0/423765377 shutdown_connections 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.050+0000 7f7d4ea26700 1 -- 172.21.15.102:0/423765377 wait complete. 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.050+0000 7f7d4ea26700 1 Processor -- start 2024-04-24T23:03:23.831 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.050+0000 7f7d4ea26700 1 -- start start 2024-04-24T23:03:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d4ea26700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d48108120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d4ea26700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4810c060 con 0x7f7d481088a0 2024-04-24T23:03:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d48108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d47fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d48108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40814/0 (socket says 172.21.15.102:40814) 2024-04-24T23:03:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d47fff700 1 -- 172.21.15.102:0/1432653995 learned_addr learned my addr 172.21.15.102:0/1432653995 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d47fff700 1 -- 172.21.15.102:0/1432653995 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d48108660 con 0x7f7d481088a0 2024-04-24T23:03:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d47fff700 1 --2- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d48108120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c011a50 tx=0x7f7d3c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d457fa700 1 -- 172.21.15.102:0/1432653995 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d3c003a80 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d457fa700 1 -- 172.21.15.102:0/1432653995 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7d3c003c20 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d457fa700 1 -- 172.21.15.102:0/1432653995 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d3c01a650 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.051+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d48106850 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.052+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d48106cd0 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.052+0000 7f7d457fa700 1 -- 172.21.15.102:0/1432653995 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f7d3c024ad0 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.053+0000 7f7d457fa700 1 --2- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7d30038f30 0x7f7d3003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.053+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d48063890 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.053+0000 7f7d457fa700 1 -- 172.21.15.102:0/1432653995 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7d3c04d3d0 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.053+0000 7f7d477fe700 1 --2- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7d30038f30 0x7f7d3003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.053+0000 7f7d477fe700 1 --2- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7d30038f30 0x7f7d3003b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006fd0 tx=0x7f7d38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.058+0000 7f7d457fa700 1 -- 172.21.15.102:0/1432653995 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d3c0172b0 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.093+0000 7f7d457fa700 1 -- 172.21.15.102:0/1432653995 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7d3c02d770 con 0x7f7d481088a0 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.277+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7d48103480 con 0x7f7d30038f30 2024-04-24T23:03:23.833 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.283+0000 7f7d457fa700 1 -- 172.21.15.102:0/1432653995 <== mgr.14120 v2:172.21.15.102:6800/3017038970 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7d48103480 con 0x7f7d30038f30 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7d30038f30 msgr2=0x7f7d3003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 --2- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7d30038f30 0x7f7d3003b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006fd0 tx=0x7f7d38006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 msgr2=0x7f7d48108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 --2- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d48108120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c011a50 tx=0x7f7d3c011b30 comp rx=0 tx=0).stop 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 shutdown_connections 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 --2- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7d30038f30 0x7f7d3003b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 --2- 172.21.15.102:0/1432653995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d481088a0 0x7f7d48108120 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 >> 172.21.15.102:0/1432653995 conn(0x7f7d481021d0 msgr2=0x7f7d48102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 shutdown_connections 2024-04-24T23:03:23.834 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:23.285+0000 7f7d4ea26700 1 -- 172.21.15.102:0/1432653995 wait complete. 2024-04-24T23:03:24.215 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:24 smithi102 ceph-mon[29216]: mgrmap e8: x(active, since 2s) 2024-04-24T23:03:24.215 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:24 smithi102 ceph-mon[29216]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:24.215 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:24 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:24.215 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:24 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:24.215 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:24 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:25.077 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.227+0000 7fb0a0fd1700 1 Processor -- start 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.227+0000 7fb0a0fd1700 1 -- start start 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.228+0000 7fb0a0fd1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.228+0000 7fb0a0fd1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c07c8b0 con 0x7fb09c07de80 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.228+0000 7fb09a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.228+0000 7fb09a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40824/0 (socket says 172.21.15.102:40824) 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.228+0000 7fb09a59c700 1 -- 172.21.15.102:0/1546453919 learned_addr learned my addr 172.21.15.102:0/1546453919 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.228+0000 7fb09a59c700 1 -- 172.21.15.102:0/1546453919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c07ca30 con 0x7fb09c07de80 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.228+0000 7fb09a59c700 1 --2- 172.21.15.102:0/1546453919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c07c2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0098a0 tx=0x7fb08c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b863246a0b1155 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb09959a700 1 -- 172.21.15.102:0/1546453919 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb08c004160 con 0x7fb09c07de80 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb09959a700 1 -- 172.21.15.102:0/1546453919 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb08c004300 con 0x7fb09c07de80 2024-04-24T23:03:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb09959a700 1 -- 172.21.15.102:0/1546453919 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb08c011a80 con 0x7fb09c07de80 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/1546453919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 msgr2=0x7fb09c07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb0a0fd1700 1 --2- 172.21.15.102:0/1546453919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c07c2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0098a0 tx=0x7fb08c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/1546453919 shutdown_connections 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb0a0fd1700 1 --2- 172.21.15.102:0/1546453919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c07c2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/1546453919 >> 172.21.15.102:0/1546453919 conn(0x7fb09c103320 msgr2=0x7fb09c105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/1546453919 shutdown_connections 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.229+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/1546453919 wait complete. 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.230+0000 7fb0a0fd1700 1 Processor -- start 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.230+0000 7fb0a0fd1700 1 -- start start 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.230+0000 7fb0a0fd1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.230+0000 7fb0a0fd1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c10e310 con 0x7fb09c07de80 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.230+0000 7fb09a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.230+0000 7fb09a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40838/0 (socket says 172.21.15.102:40838) 2024-04-24T23:03:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.230+0000 7fb09a59c700 1 -- 172.21.15.102:0/2759255404 learned_addr learned my addr 172.21.15.102:0/2759255404 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.231+0000 7fb09a59c700 1 -- 172.21.15.102:0/2759255404 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c19e3b0 con 0x7fb09c07de80 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.231+0000 7fb09a59c700 1 --2- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c19de70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb08c006af0 tx=0x7fb08c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.231+0000 7fb08b7fe700 1 -- 172.21.15.102:0/2759255404 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb08c003930 con 0x7fb09c07de80 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.231+0000 7fb08b7fe700 1 -- 172.21.15.102:0/2759255404 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb08c003ad0 con 0x7fb09c07de80 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.231+0000 7fb08b7fe700 1 -- 172.21.15.102:0/2759255404 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb08c0195e0 con 0x7fb09c07de80 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.231+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c19e640 con 0x7fb09c07de80 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.231+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09c1a1330 con 0x7fb09c07de80 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.232+0000 7fb08b7fe700 1 -- 172.21.15.102:0/2759255404 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb08c019890 con 0x7fb09c07de80 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.232+0000 7fb08b7fe700 1 --2- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fb084039050 0x7fb08403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.232+0000 7fb08b7fe700 1 -- 172.21.15.102:0/2759255404 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb08c04d8a0 con 0x7fb09c07de80 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.232+0000 7fb099d9b700 1 --2- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fb084039050 0x7fb08403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.232+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb09c063890 con 0x7fb09c07de80 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.232+0000 7fb099d9b700 1 --2- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fb084039050 0x7fb08403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb090006fd0 tx=0x7fb090006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:25.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.237+0000 7fb08b7fe700 1 -- 172.21.15.102:0/2759255404 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb08c00fda0 con 0x7fb09c07de80 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.435+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb09c1045f0 con 0x7fb084039050 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.436+0000 7fb08b7fe700 1 -- 172.21.15.102:0/2759255404 <== mgr.14120 v2:172.21.15.102:6800/3017038970 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb09c1045f0 con 0x7fb084039050 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.438+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fb084039050 msgr2=0x7fb08403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.438+0000 7fb0a0fd1700 1 --2- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fb084039050 0x7fb08403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb090006fd0 tx=0x7fb090006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.438+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 msgr2=0x7fb09c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.438+0000 7fb0a0fd1700 1 --2- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c19de70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb08c006af0 tx=0x7fb08c011b70 comp rx=0 tx=0).stop 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.438+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 shutdown_connections 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.438+0000 7fb0a0fd1700 1 --2- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fb084039050 0x7fb08403b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.438+0000 7fb0a0fd1700 1 --2- 172.21.15.102:0/2759255404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb09c07de80 0x7fb09c19de70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.438+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 >> 172.21.15.102:0/2759255404 conn(0x7fb09c103320 msgr2=0x7fb09c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.438+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 shutdown_connections 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:24.439+0000 7fb0a0fd1700 1 -- 172.21.15.102:0/2759255404 wait complete. 2024-04-24T23:03:25.081 INFO:teuthology.orchestra.run.smithi102.stdout:Generating ssh key... 2024-04-24T23:03:25.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-mon[29216]: [24/Apr/2024:23:03:23] ENGINE Bus STARTING 2024-04-24T23:03:25.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-mon[29216]: [24/Apr/2024:23:03:23] ENGINE Serving on https://172.21.15.102:7150 2024-04-24T23:03:25.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-mon[29216]: [24/Apr/2024:23:03:23] ENGINE Serving on http://172.21.15.102:8765 2024-04-24T23:03:25.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-mon[29216]: [24/Apr/2024:23:03:23] ENGINE Bus STARTED 2024-04-24T23:03:26.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: Generating public/private rsa key pair. 2024-04-24T23:03:26.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: Your identification has been saved in /tmp/tmpq7bd_zd6/key. 2024-04-24T23:03:26.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: Your public key has been saved in /tmp/tmpq7bd_zd6/key.pub. 2024-04-24T23:03:26.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: The key fingerprint is: 2024-04-24T23:03:26.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: SHA256:H1Hewowuxhe0xtrOm2bGNjXjcf7J9EzGw2tf4661lzE ceph-97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:26.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: The key's randomart image is: 2024-04-24T23:03:26.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: +---[RSA 3072]----+ 2024-04-24T23:03:26.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: | . . | 2024-04-24T23:03:26.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: | o B . | 2024-04-24T23:03:26.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: | B = . | 2024-04-24T23:03:26.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: | . = o . | 2024-04-24T23:03:26.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: | S = | 2024-04-24T23:03:26.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: | . * .= oE | 2024-04-24T23:03:26.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: | .+o * *O| 2024-04-24T23:03:26.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: | Bo. **X| 2024-04-24T23:03:26.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: | =o. .+OB| 2024-04-24T23:03:26.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: +----[SHA256]-----+ 2024-04-24T23:03:26.475 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:26 smithi102 ceph-mon[29216]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:26.475 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:26 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:26.476 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:26 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:26.484 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.458+0000 7f47f11b5700 1 Processor -- start 2024-04-24T23:03:26.484 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.458+0000 7f47f11b5700 1 -- start start 2024-04-24T23:03:26.484 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.458+0000 7f47f11b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec108210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:26.484 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.458+0000 7f47f11b5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1087e0 con 0x7f47ec107df0 2024-04-24T23:03:26.484 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.458+0000 7f47ead9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:26.484 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.458+0000 7f47ead9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40840/0 (socket says 172.21.15.102:40840) 2024-04-24T23:03:26.484 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.458+0000 7f47ead9d700 1 -- 172.21.15.102:0/3655983265 learned_addr learned my addr 172.21.15.102:0/3655983265 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:26.484 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47ead9d700 1 -- 172.21.15.102:0/3655983265 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec108ff0 con 0x7f47ec107df0 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47ead9d700 1 --2- 172.21.15.102:0/3655983265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec108210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0098a0 tx=0x7f47dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e0a343727b7486 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47e9d9b700 1 -- 172.21.15.102:0/3655983265 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47dc004160 con 0x7f47ec107df0 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47e9d9b700 1 -- 172.21.15.102:0/3655983265 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f47dc004300 con 0x7f47ec107df0 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47e9d9b700 1 -- 172.21.15.102:0/3655983265 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47dc011a80 con 0x7f47ec107df0 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47f11b5700 1 -- 172.21.15.102:0/3655983265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 msgr2=0x7f47ec108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47f11b5700 1 --2- 172.21.15.102:0/3655983265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec108210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0098a0 tx=0x7f47dc009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47f11b5700 1 -- 172.21.15.102:0/3655983265 shutdown_connections 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47f11b5700 1 --2- 172.21.15.102:0/3655983265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec108210 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47f11b5700 1 -- 172.21.15.102:0/3655983265 >> 172.21.15.102:0/3655983265 conn(0x7f47ec103370 msgr2=0x7f47ec1057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47f11b5700 1 -- 172.21.15.102:0/3655983265 shutdown_connections 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.459+0000 7f47f11b5700 1 -- 172.21.15.102:0/3655983265 wait complete. 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.460+0000 7f47f11b5700 1 Processor -- start 2024-04-24T23:03:26.485 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.460+0000 7f47f11b5700 1 -- start start 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.460+0000 7f47f11b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.460+0000 7f47f11b5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec10c220 con 0x7f47ec107df0 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47ead9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47ead9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec19bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40842/0 (socket says 172.21.15.102:40842) 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47ead9d700 1 -- 172.21.15.102:0/668941243 learned_addr learned my addr 172.21.15.102:0/668941243 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47ead9d700 1 -- 172.21.15.102:0/668941243 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec19c220 con 0x7f47ec107df0 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47ead9d700 1 --2- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec19bce0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f47dc006af0 tx=0x7f47dc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47d3fff700 1 -- 172.21.15.102:0/668941243 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47dc003930 con 0x7f47ec107df0 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47d3fff700 1 -- 172.21.15.102:0/668941243 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f47dc003ad0 con 0x7f47ec107df0 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec19c4b0 con 0x7f47ec107df0 2024-04-24T23:03:26.486 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47d3fff700 1 -- 172.21.15.102:0/668941243 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47dc0195e0 con 0x7f47ec107df0 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.461+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec19f190 con 0x7f47ec107df0 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.462+0000 7f47d3fff700 1 -- 172.21.15.102:0/668941243 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f47dc0198e0 con 0x7f47ec107df0 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.462+0000 7f47d3fff700 1 --2- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f47d4039000 0x7f47d403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.462+0000 7f47d3fff700 1 -- 172.21.15.102:0/668941243 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f47dc04e950 con 0x7f47ec107df0 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.462+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ec063890 con 0x7f47ec107df0 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.462+0000 7f47ea59c700 1 --2- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f47d4039000 0x7f47d403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.463+0000 7f47ea59c700 1 --2- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f47d4039000 0x7f47d403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47e0006fd0 tx=0x7f47e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.467+0000 7f47d3fff700 1 -- 172.21.15.102:0/668941243 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f47dc023390 con 0x7f47ec107df0 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.662+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f47ec104670 con 0x7f47d4039000 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.877+0000 7f47d3fff700 1 -- 172.21.15.102:0/668941243 <== mgr.14120 v2:172.21.15.102:6800/3017038970 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f47ec104670 con 0x7f47d4039000 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.879+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f47d4039000 msgr2=0x7f47d403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.879+0000 7f47f11b5700 1 --2- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f47d4039000 0x7f47d403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47e0006fd0 tx=0x7f47e0006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:26.487 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.879+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 msgr2=0x7f47ec19bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:26.488 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.879+0000 7f47f11b5700 1 --2- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec19bce0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f47dc006af0 tx=0x7f47dc011b70 comp rx=0 tx=0).stop 2024-04-24T23:03:26.488 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.879+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 shutdown_connections 2024-04-24T23:03:26.488 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.879+0000 7f47f11b5700 1 --2- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f47d4039000 0x7f47d403b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:26.488 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.879+0000 7f47f11b5700 1 --2- 172.21.15.102:0/668941243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47ec107df0 0x7f47ec19bce0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:26.488 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.879+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 >> 172.21.15.102:0/668941243 conn(0x7f47ec103370 msgr2=0x7f47ec103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:26.488 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.880+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 shutdown_connections 2024-04-24T23:03:26.488 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:25.880+0000 7f47f11b5700 1 -- 172.21.15.102:0/668941243 wait complete. 2024-04-24T23:03:27.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:27 smithi102 ceph-mon[29216]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:27.575 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:27 smithi102 ceph-mon[29216]: Generating ssh key... 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4/oogTrwYQqZGuFhFGaoC/8pV3KBalx4+pwkGkE+r3kww0yOp49llyx2ROboMyRlwufZsWIy2cEG0Z7B+D3DE3st3LMdC9jWSv6BQzVU5T9bJgWECrqLP51VfWvMy/crzor3tbNY0wLx2f77u+ACTsELzKVOWcv3EEvO3jaZissFQ3Pk2L2K4zPv/LV8eObNs6v8q5NBEhfSqoboF3jD7rJQd4IpNT8CnsmaifgjSm1sLkFjsDs0vcYBamtQ4bQg1uGUA0RZ8Hj2fe63kwnhXaM069qn5LkbhQA9QfLycnRxBunUGq1OTSVZJYW2i4RDVvlI5pQjx+GF4HsQ4ElNc2WlWjRnEMHkEQr/z0+yjjCWoT0UuQgx5c+rRjqZ59vv8UKas9eyjkRspIt+yxkZ3hVi+7lbofi3srNPE72ODkuM4eokQQ5r81TQAU7sZ7eky8E2rOMvHCR6No44z0gfyBA4ObuTGXr0jIPkLVOC3ZHGqLHq4nJbSVvrWsjvvw0= ceph-97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.877+0000 7f3be776d700 1 Processor -- start 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.877+0000 7f3be776d700 1 -- start start 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.877+0000 7f3be776d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.877+0000 7f3be776d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3be007c8f0 con 0x7f3be007dec0 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.878+0000 7f3be5509700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.878+0000 7f3be5509700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39118/0 (socket says 172.21.15.102:39118) 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.878+0000 7f3be5509700 1 -- 172.21.15.102:0/1890001889 learned_addr learned my addr 172.21.15.102:0/1890001889 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.878+0000 7f3be5509700 1 -- 172.21.15.102:0/1890001889 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be007ca70 con 0x7f3be007dec0 2024-04-24T23:03:27.673 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.878+0000 7f3be5509700 1 --2- 172.21.15.102:0/1890001889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be007c320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0098a0 tx=0x7f3bdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717ce5eb37c37395 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.878+0000 7f3bd7fff700 1 -- 172.21.15.102:0/1890001889 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bdc004160 con 0x7f3be007dec0 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.878+0000 7f3bd7fff700 1 -- 172.21.15.102:0/1890001889 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3bdc004300 con 0x7f3be007dec0 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.878+0000 7f3bd7fff700 1 -- 172.21.15.102:0/1890001889 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bdc011a80 con 0x7f3be007dec0 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.879+0000 7f3be776d700 1 -- 172.21.15.102:0/1890001889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 msgr2=0x7f3be007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.879+0000 7f3be776d700 1 --2- 172.21.15.102:0/1890001889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be007c320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0098a0 tx=0x7f3bdc009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.879+0000 7f3be776d700 1 -- 172.21.15.102:0/1890001889 shutdown_connections 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.879+0000 7f3be776d700 1 --2- 172.21.15.102:0/1890001889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be007c320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.879+0000 7f3be776d700 1 -- 172.21.15.102:0/1890001889 >> 172.21.15.102:0/1890001889 conn(0x7f3be0103320 msgr2=0x7f3be0105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.879+0000 7f3be776d700 1 -- 172.21.15.102:0/1890001889 shutdown_connections 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.879+0000 7f3be776d700 1 -- 172.21.15.102:0/1890001889 wait complete. 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.879+0000 7f3be776d700 1 Processor -- start 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.879+0000 7f3be776d700 1 -- start start 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3be776d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:27.674 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3be776d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3be010e290 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3be5509700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3be5509700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be019dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39134/0 (socket says 172.21.15.102:39134) 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3be5509700 1 -- 172.21.15.102:0/1666137617 learned_addr learned my addr 172.21.15.102:0/1666137617 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3be5509700 1 -- 172.21.15.102:0/1666137617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be019e400 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3be5509700 1 --2- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be019dec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc000c00 tx=0x7f3bdc004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3bd67fc700 1 -- 172.21.15.102:0/1666137617 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bdc0039c0 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3bd67fc700 1 -- 172.21.15.102:0/1666137617 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3bdc003b60 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be019e690 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3bd67fc700 1 -- 172.21.15.102:0/1666137617 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bdc01a6f0 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.880+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be01a1380 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.881+0000 7f3bd67fc700 1 -- 172.21.15.102:0/1666137617 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3bdc01a9a0 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.881+0000 7f3bd67fc700 1 --2- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f3bcc039050 0x7f3bcc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.881+0000 7f3bd67fc700 1 -- 172.21.15.102:0/1666137617 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3bdc04d360 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.881+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be0063890 con 0x7f3be007dec0 2024-04-24T23:03:27.675 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.881+0000 7f3be4d08700 1 --2- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f3bcc039050 0x7f3bcc03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.882+0000 7f3be4d08700 1 --2- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f3bcc039050 0x7f3bcc03b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0006fd0 tx=0x7f3bd0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:26.886+0000 7f3bd67fc700 1 -- 172.21.15.102:0/1666137617 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3bdc020240 con 0x7f3be007dec0 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.088+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f3be0104570 con 0x7f3bcc039050 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.088+0000 7f3bd67fc700 1 -- 172.21.15.102:0/1666137617 <== mgr.14120 v2:172.21.15.102:6800/3017038970 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f3be0104570 con 0x7f3bcc039050 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f3bcc039050 msgr2=0x7f3bcc03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 --2- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f3bcc039050 0x7f3bcc03b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0006fd0 tx=0x7f3bd0006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 msgr2=0x7f3be019dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 --2- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be019dec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc000c00 tx=0x7f3bdc004290 comp rx=0 tx=0).stop 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 shutdown_connections 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 --2- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f3bcc039050 0x7f3bcc03b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 --2- 172.21.15.102:0/1666137617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3be007dec0 0x7f3be019dec0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 >> 172.21.15.102:0/1666137617 conn(0x7f3be0103320 msgr2=0x7f3be0103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 shutdown_connections 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:27.090+0000 7f3be776d700 1 -- 172.21.15.102:0/1666137617 wait complete. 2024-04-24T23:03:27.676 INFO:teuthology.orchestra.run.smithi102.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:03:27.677 INFO:teuthology.orchestra.run.smithi102.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T23:03:27.677 INFO:teuthology.orchestra.run.smithi102.stdout:Adding host smithi102... 2024-04-24T23:03:28.304 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:28 smithi102 ceph-mon[29216]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:29.179 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:29 smithi102 ceph-mon[29216]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "addr": "172.21.15.102", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:30.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:30 smithi102 ceph-mon[29216]: Deploying cephadm binary to smithi102 2024-04-24T23:03:33.156 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Added host 'smithi102' with addr '172.21.15.102' 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.094+0000 7f0df4cf3700 1 Processor -- start 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.094+0000 7f0df4cf3700 1 -- start start 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.094+0000 7f0df4cf3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df01081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.094+0000 7f0df4cf3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0df0108770 con 0x7f0df0107d80 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.094+0000 7f0dee59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df01081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.094+0000 7f0dee59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df01081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39144/0 (socket says 172.21.15.102:39144) 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.094+0000 7f0dee59c700 1 -- 172.21.15.102:0/4286690042 learned_addr learned my addr 172.21.15.102:0/4286690042 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.095+0000 7f0dee59c700 1 -- 172.21.15.102:0/4286690042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df0108f80 con 0x7f0df0107d80 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.095+0000 7f0dee59c700 1 --2- 172.21.15.102:0/4286690042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df01081a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0de00098a0 tx=0x7f0de0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b31d75c3db5400 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.095+0000 7f0ded59a700 1 -- 172.21.15.102:0/4286690042 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de0004160 con 0x7f0df0107d80 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.095+0000 7f0ded59a700 1 -- 172.21.15.102:0/4286690042 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0de0004300 con 0x7f0df0107d80 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.095+0000 7f0ded59a700 1 -- 172.21.15.102:0/4286690042 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de0011a80 con 0x7f0df0107d80 2024-04-24T23:03:33.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.096+0000 7f0df4cf3700 1 -- 172.21.15.102:0/4286690042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 msgr2=0x7f0df01081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.096+0000 7f0df4cf3700 1 --2- 172.21.15.102:0/4286690042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df01081a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0de00098a0 tx=0x7f0de0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.096+0000 7f0df4cf3700 1 -- 172.21.15.102:0/4286690042 shutdown_connections 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.096+0000 7f0df4cf3700 1 --2- 172.21.15.102:0/4286690042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df01081a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.096+0000 7f0df4cf3700 1 -- 172.21.15.102:0/4286690042 >> 172.21.15.102:0/4286690042 conn(0x7f0df0103320 msgr2=0x7f0df0105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.096+0000 7f0df4cf3700 1 -- 172.21.15.102:0/4286690042 shutdown_connections 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.096+0000 7f0df4cf3700 1 -- 172.21.15.102:0/4286690042 wait complete. 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.096+0000 7f0df4cf3700 1 Processor -- start 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.096+0000 7f0df4cf3700 1 -- start start 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0df4cf3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0df4cf3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0df010c1b0 con 0x7f0df0107d80 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0dee59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0dee59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df019bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39158/0 (socket says 172.21.15.102:39158) 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0dee59c700 1 -- 172.21.15.102:0/2675690253 learned_addr learned my addr 172.21.15.102:0/2675690253 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:33.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0dee59c700 1 -- 172.21.15.102:0/2675690253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df019c220 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0dee59c700 1 --2- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df019bce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0de0000c00 tx=0x7f0de0004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0ddf7fe700 1 -- 172.21.15.102:0/2675690253 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de00039c0 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0ddf7fe700 1 -- 172.21.15.102:0/2675690253 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0de0003b60 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0ddf7fe700 1 -- 172.21.15.102:0/2675690253 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de001a6f0 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df019c4b0 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.097+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df019f190 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.098+0000 7f0ddf7fe700 1 -- 172.21.15.102:0/2675690253 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0de001a9a0 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.098+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df010c330 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.098+0000 7f0ddf7fe700 1 --2- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0dd8039050 0x7f0dd803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.098+0000 7f0ddf7fe700 1 -- 172.21.15.102:0/2675690253 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0de004d540 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.098+0000 7f0dedd9b700 1 --2- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0dd8039050 0x7f0dd803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.099+0000 7f0dedd9b700 1 --2- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0dd8039050 0x7f0dd803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006fd0 tx=0x7f0de4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.103+0000 7f0ddf7fe700 1 -- 172.21.15.102:0/2675690253 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0de000bc40 con 0x7f0df0107d80 2024-04-24T23:03:33.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:28.302+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "addr": "172.21.15.102", "target": ["mon-mgr", ""]}) -- 0x7f0df0104630 con 0x7f0dd8039050 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.450+0000 7f0ddf7fe700 1 -- 172.21.15.102:0/2675690253 <== mgr.14120 v2:172.21.15.102:6800/3017038970 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0df0104630 con 0x7f0dd8039050 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0dd8039050 msgr2=0x7f0dd803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 --2- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0dd8039050 0x7f0dd803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006fd0 tx=0x7f0de4006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 msgr2=0x7f0df019bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 --2- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df019bce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0de0000c00 tx=0x7f0de0004290 comp rx=0 tx=0).stop 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 shutdown_connections 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 --2- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0dd8039050 0x7f0dd803b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 --2- 172.21.15.102:0/2675690253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0df0107d80 0x7f0df019bce0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 >> 172.21.15.102:0/2675690253 conn(0x7f0df0103320 msgr2=0x7f0df0103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 shutdown_connections 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:32.453+0000 7f0df4cf3700 1 -- 172.21.15.102:0/2675690253 wait complete. 2024-04-24T23:03:33.160 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying unmanaged mon service... 2024-04-24T23:03:33.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:33 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:33.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:33 smithi102 ceph-mon[29216]: Added host smithi102 2024-04-24T23:03:33.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:33 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:34.560 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T23:03:34.563 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.604+0000 7f0e4c030700 1 Processor -- start 2024-04-24T23:03:34.563 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.605+0000 7f0e4c030700 1 -- start start 2024-04-24T23:03:34.563 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.605+0000 7f0e4c030700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e441070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:34.563 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.605+0000 7f0e4c030700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441076b0 con 0x7f0e44106cc0 2024-04-24T23:03:34.563 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.605+0000 7f0e49dcc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e441070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:34.563 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.605+0000 7f0e49dcc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e441070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39166/0 (socket says 172.21.15.102:39166) 2024-04-24T23:03:34.563 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.605+0000 7f0e49dcc700 1 -- 172.21.15.102:0/4098922088 learned_addr learned my addr 172.21.15.102:0/4098922088 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.605+0000 7f0e49dcc700 1 -- 172.21.15.102:0/4098922088 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e44107ec0 con 0x7f0e44106cc0 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e49dcc700 1 --2- 172.21.15.102:0/4098922088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e441070e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e340098a0 tx=0x7f0e34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a90375bda0567ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e48dca700 1 -- 172.21.15.102:0/4098922088 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e34004160 con 0x7f0e44106cc0 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e48dca700 1 -- 172.21.15.102:0/4098922088 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0e34004300 con 0x7f0e44106cc0 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e48dca700 1 -- 172.21.15.102:0/4098922088 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e34011a80 con 0x7f0e44106cc0 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e4c030700 1 -- 172.21.15.102:0/4098922088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 msgr2=0x7f0e441070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e4c030700 1 --2- 172.21.15.102:0/4098922088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e441070e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e340098a0 tx=0x7f0e34009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e4c030700 1 -- 172.21.15.102:0/4098922088 shutdown_connections 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e4c030700 1 --2- 172.21.15.102:0/4098922088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e441070e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e4c030700 1 -- 172.21.15.102:0/4098922088 >> 172.21.15.102:0/4098922088 conn(0x7f0e44102240 msgr2=0x7f0e441046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.606+0000 7f0e4c030700 1 -- 172.21.15.102:0/4098922088 shutdown_connections 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.607+0000 7f0e4c030700 1 -- 172.21.15.102:0/4098922088 wait complete. 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.607+0000 7f0e4c030700 1 Processor -- start 2024-04-24T23:03:34.564 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.607+0000 7f0e4c030700 1 -- start start 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.607+0000 7f0e4c030700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e4419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.607+0000 7f0e4c030700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4410b0f0 con 0x7f0e44106cc0 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.607+0000 7f0e49dcc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e4419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.607+0000 7f0e49dcc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e4419bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39178/0 (socket says 172.21.15.102:39178) 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.607+0000 7f0e49dcc700 1 -- 172.21.15.102:0/2858388283 learned_addr learned my addr 172.21.15.102:0/2858388283 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.608+0000 7f0e49dcc700 1 -- 172.21.15.102:0/2858388283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4419c230 con 0x7f0e44106cc0 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.608+0000 7f0e49dcc700 1 --2- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e4419bcf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e34000c00 tx=0x7f0e34011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.608+0000 7f0e3affd700 1 -- 172.21.15.102:0/2858388283 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e340039f0 con 0x7f0e44106cc0 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.608+0000 7f0e3affd700 1 -- 172.21.15.102:0/2858388283 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0e34003b90 con 0x7f0e44106cc0 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.608+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4419c4c0 con 0x7f0e44106cc0 2024-04-24T23:03:34.568 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.608+0000 7f0e3affd700 1 -- 172.21.15.102:0/2858388283 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e3401a740 con 0x7f0e44106cc0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.608+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4419f1a0 con 0x7f0e44106cc0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.609+0000 7f0e3affd700 1 -- 172.21.15.102:0/2858388283 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0e34024440 con 0x7f0e44106cc0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.609+0000 7f0e3affd700 1 --2- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e300390a0 0x7f0e3003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.609+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e4410b270 con 0x7f0e44106cc0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.609+0000 7f0e3affd700 1 -- 172.21.15.102:0/2858388283 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0e3404d3b0 con 0x7f0e44106cc0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.609+0000 7f0e495cb700 1 --2- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e300390a0 0x7f0e3003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.609+0000 7f0e495cb700 1 --2- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e300390a0 0x7f0e3003b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e40006fd0 tx=0x7f0e40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.614+0000 7f0e3affd700 1 -- 172.21.15.102:0/2858388283 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0e34024730 con 0x7f0e44106cc0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.852+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0e4402de30 con 0x7f0e300390a0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.860+0000 7f0e3affd700 1 -- 172.21.15.102:0/2858388283 <== mgr.14120 v2:172.21.15.102:6800/3017038970 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0e4402de30 con 0x7f0e300390a0 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e300390a0 msgr2=0x7f0e3003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 --2- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e300390a0 0x7f0e3003b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e40006fd0 tx=0x7f0e40006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 msgr2=0x7f0e4419bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:34.569 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 --2- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e4419bcf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e34000c00 tx=0x7f0e34011b00 comp rx=0 tx=0).stop 2024-04-24T23:03:34.570 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 shutdown_connections 2024-04-24T23:03:34.570 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 --2- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f0e300390a0 0x7f0e3003b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:34.570 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 --2- 172.21.15.102:0/2858388283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e44106cc0 0x7f0e4419bcf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:34.570 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 >> 172.21.15.102:0/2858388283 conn(0x7f0e44102240 msgr2=0x7f0e44103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:34.570 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 shutdown_connections 2024-04-24T23:03:34.570 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:33.862+0000 7f0e4c030700 1 -- 172.21.15.102:0/2858388283 wait complete. 2024-04-24T23:03:34.570 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying unmanaged mgr service... 2024-04-24T23:03:35.129 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:34 smithi102 ceph-mon[29216]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:35.129 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:34 smithi102 ceph-mon[29216]: Saving service mon spec with placement count:5 2024-04-24T23:03:35.130 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:34 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:36.796 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T23:03:36.796 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.027+0000 7f81ddd05700 1 Processor -- start 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.027+0000 7f81ddd05700 1 -- start start 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81ddd05700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81ddd05700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81d807c890 con 0x7f81d807de60 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81d77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81d77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39182/0 (socket says 172.21.15.102:39182) 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81d77fe700 1 -- 172.21.15.102:0/976711081 learned_addr learned my addr 172.21.15.102:0/976711081 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81d77fe700 1 -- 172.21.15.102:0/976711081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d807ca10 con 0x7f81d807de60 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81d77fe700 1 --2- 172.21.15.102:0/976711081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d807c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f81c80098a0 tx=0x7f81c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509682ab5060e156 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81d67fc700 1 -- 172.21.15.102:0/976711081 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81c8004160 con 0x7f81d807de60 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81d67fc700 1 -- 172.21.15.102:0/976711081 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81c8004300 con 0x7f81d807de60 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.028+0000 7f81d67fc700 1 -- 172.21.15.102:0/976711081 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81c8011a50 con 0x7f81d807de60 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.029+0000 7f81ddd05700 1 -- 172.21.15.102:0/976711081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 msgr2=0x7f81d807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:36.797 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.029+0000 7f81ddd05700 1 --2- 172.21.15.102:0/976711081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d807c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f81c80098a0 tx=0x7f81c8009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.029+0000 7f81ddd05700 1 -- 172.21.15.102:0/976711081 shutdown_connections 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.029+0000 7f81ddd05700 1 --2- 172.21.15.102:0/976711081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d807c2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.029+0000 7f81ddd05700 1 -- 172.21.15.102:0/976711081 >> 172.21.15.102:0/976711081 conn(0x7f81d81032e0 msgr2=0x7f81d8105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.029+0000 7f81ddd05700 1 -- 172.21.15.102:0/976711081 shutdown_connections 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.029+0000 7f81ddd05700 1 -- 172.21.15.102:0/976711081 wait complete. 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.030+0000 7f81ddd05700 1 Processor -- start 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.030+0000 7f81ddd05700 1 -- start start 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.030+0000 7f81ddd05700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.030+0000 7f81ddd05700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81d810e320 con 0x7f81d807de60 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.030+0000 7f81d77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:36.798 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.030+0000 7f81d77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39198/0 (socket says 172.21.15.102:39198) 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.030+0000 7f81d77fe700 1 -- 172.21.15.102:0/1666279237 learned_addr learned my addr 172.21.15.102:0/1666279237 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.030+0000 7f81d77fe700 1 -- 172.21.15.102:0/1666279237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d819e340 con 0x7f81d807de60 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.031+0000 7f81d77fe700 1 --2- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d819de00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81c8011a20 tx=0x7f81c8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.031+0000 7f81d4ff9700 1 -- 172.21.15.102:0/1666279237 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81c8003b70 con 0x7f81d807de60 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.031+0000 7f81d4ff9700 1 -- 172.21.15.102:0/1666279237 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81c801a8e0 con 0x7f81d807de60 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.031+0000 7f81d4ff9700 1 -- 172.21.15.102:0/1666279237 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81c8024990 con 0x7f81d807de60 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.031+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d819e5d0 con 0x7f81d807de60 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.031+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d81a12c0 con 0x7f81d807de60 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.032+0000 7f81d4ff9700 1 -- 172.21.15.102:0/1666279237 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f81c8022020 con 0x7f81d807de60 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.032+0000 7f81d4ff9700 1 --2- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f81c0039050 0x7f81c003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.032+0000 7f81d4ff9700 1 -- 172.21.15.102:0/1666279237 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f81c804e740 con 0x7f81d807de60 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.032+0000 7f81d6ffd700 1 --2- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f81c0039050 0x7f81c003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.032+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d8063890 con 0x7f81d807de60 2024-04-24T23:03:36.799 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.032+0000 7f81d6ffd700 1 --2- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f81c0039050 0x7f81c003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006fd0 tx=0x7f81cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.037+0000 7f81d4ff9700 1 -- 172.21.15.102:0/1666279237 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f81c800b5a0 con 0x7f81d807de60 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.262+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f81d81044f0 con 0x7f81c0039050 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.270+0000 7f81d4ff9700 1 -- 172.21.15.102:0/1666279237 <== mgr.14120 v2:172.21.15.102:6800/3017038970 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f81d81044f0 con 0x7f81c0039050 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.272+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f81c0039050 msgr2=0x7f81c003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.272+0000 7f81ddd05700 1 --2- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f81c0039050 0x7f81c003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006fd0 tx=0x7f81cc006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.272+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 msgr2=0x7f81d819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.272+0000 7f81ddd05700 1 --2- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d819de00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81c8011a20 tx=0x7f81c8011b00 comp rx=0 tx=0).stop 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.273+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 shutdown_connections 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.273+0000 7f81ddd05700 1 --2- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f81c0039050 0x7f81c003b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.273+0000 7f81ddd05700 1 --2- 172.21.15.102:0/1666279237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81d807de60 0x7f81d819de00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.273+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 >> 172.21.15.102:0/1666279237 conn(0x7f81d81032e0 msgr2=0x7f81d8103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.273+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 shutdown_connections 2024-04-24T23:03:36.800 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:36.273+0000 7f81ddd05700 1 -- 172.21.15.102:0/1666279237 wait complete. 2024-04-24T23:03:37.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:36 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:36 smithi102 ceph-mon[29216]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:36 smithi102 ceph-mon[29216]: Saving service mgr spec with placement count:2 2024-04-24T23:03:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:36 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:36 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:38.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.216+0000 7fecec4da700 1 Processor -- start 2024-04-24T23:03:38.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.216+0000 7fecec4da700 1 -- start start 2024-04-24T23:03:38.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.216+0000 7fecec4da700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece4107150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:38.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.216+0000 7fecec4da700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fece4107720 con 0x7fece4106d30 2024-04-24T23:03:38.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.216+0000 7fecea276700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece4107150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:38.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.216+0000 7fecea276700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece4107150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50292/0 (socket says 172.21.15.102:50292) 2024-04-24T23:03:38.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.216+0000 7fecea276700 1 -- 172.21.15.102:0/1192247704 learned_addr learned my addr 172.21.15.102:0/1192247704 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:38.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.217+0000 7fecea276700 1 -- 172.21.15.102:0/1192247704 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece4107f30 con 0x7fece4106d30 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.217+0000 7fecea276700 1 --2- 172.21.15.102:0/1192247704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece4107150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fece00098a0 tx=0x7fece0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce8927e9e3424ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.217+0000 7fece9274700 1 -- 172.21.15.102:0/1192247704 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece0004160 con 0x7fece4106d30 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.217+0000 7fece9274700 1 -- 172.21.15.102:0/1192247704 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fece0004300 con 0x7fece4106d30 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.217+0000 7fece9274700 1 -- 172.21.15.102:0/1192247704 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece0011a50 con 0x7fece4106d30 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.217+0000 7fecec4da700 1 -- 172.21.15.102:0/1192247704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 msgr2=0x7fece4107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.217+0000 7fecec4da700 1 --2- 172.21.15.102:0/1192247704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece4107150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fece00098a0 tx=0x7fece0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecec4da700 1 -- 172.21.15.102:0/1192247704 shutdown_connections 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecec4da700 1 --2- 172.21.15.102:0/1192247704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece4107150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecec4da700 1 -- 172.21.15.102:0/1192247704 >> 172.21.15.102:0/1192247704 conn(0x7fece4102310 msgr2=0x7fece4104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecec4da700 1 -- 172.21.15.102:0/1192247704 shutdown_connections 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecec4da700 1 -- 172.21.15.102:0/1192247704 wait complete. 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecec4da700 1 Processor -- start 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecec4da700 1 -- start start 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecec4da700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:38.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecec4da700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fece410b2e0 con 0x7fece4106d30 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecea276700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecea276700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50298/0 (socket says 172.21.15.102:50298) 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.218+0000 7fecea276700 1 -- 172.21.15.102:0/2685197758 learned_addr learned my addr 172.21.15.102:0/2685197758 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.219+0000 7fecea276700 1 -- 172.21.15.102:0/2685197758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece419b0c0 con 0x7fece4106d30 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.219+0000 7fecea276700 1 --2- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece419ab80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fece0011710 tx=0x7fece0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.219+0000 7fecdb7fe700 1 -- 172.21.15.102:0/2685197758 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece00039c0 con 0x7fece4106d30 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.219+0000 7fecdb7fe700 1 -- 172.21.15.102:0/2685197758 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fece0003b60 con 0x7fece4106d30 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.219+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece419b350 con 0x7fece4106d30 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.219+0000 7fecdb7fe700 1 -- 172.21.15.102:0/2685197758 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece001a690 con 0x7fece4106d30 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.219+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece419e040 con 0x7fece4106d30 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.220+0000 7fecdb7fe700 1 -- 172.21.15.102:0/2685197758 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fece001a990 con 0x7fece4106d30 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.220+0000 7fecdb7fe700 1 --2- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fecd0039000 0x7fecd003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:38.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.220+0000 7fecdb7fe700 1 -- 172.21.15.102:0/2685197758 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fece004d9d0 con 0x7fece4106d30 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.220+0000 7fece9a75700 1 --2- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fecd0039000 0x7fecd003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.220+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece4063890 con 0x7fece4106d30 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.220+0000 7fece9a75700 1 --2- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fecd0039000 0x7fecd003b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fecd4006fd0 tx=0x7fecd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.225+0000 7fecdb7fe700 1 -- 172.21.15.102:0/2685197758 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fece0019c70 con 0x7fece4106d30 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.438+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fece419e590 con 0x7fece4106d30 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.441+0000 7fecdb7fe700 1 -- 172.21.15.102:0/2685197758 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fece0016df0 con 0x7fece4106d30 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.443+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fecd0039000 msgr2=0x7fecd003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.443+0000 7fecec4da700 1 --2- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fecd0039000 0x7fecd003b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fecd4006fd0 tx=0x7fecd4006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.443+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 msgr2=0x7fece419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.443+0000 7fecec4da700 1 --2- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece419ab80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fece0011710 tx=0x7fece0011ad0 comp rx=0 tx=0).stop 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.444+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 shutdown_connections 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.444+0000 7fecec4da700 1 --2- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fecd0039000 0x7fecd003b4c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.444+0000 7fecec4da700 1 --2- 172.21.15.102:0/2685197758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fece4106d30 0x7fece419ab80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:38.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.444+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 >> 172.21.15.102:0/2685197758 conn(0x7fece4102310 msgr2=0x7fece4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:38.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.444+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 shutdown_connections 2024-04-24T23:03:38.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:37.444+0000 7fecec4da700 1 -- 172.21.15.102:0/2685197758 wait complete. 2024-04-24T23:03:38.699 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:38 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2685197758' entity='client.admin' 2024-04-24T23:03:38.699 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:38 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:39.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.005+0000 7ff469eca700 1 Processor -- start 2024-04-24T23:03:39.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.005+0000 7ff469eca700 1 -- start start 2024-04-24T23:03:39.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.005+0000 7ff469eca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff4641067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:39.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.005+0000 7ff469eca700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff464106d90 con 0x7ff4641063a0 2024-04-24T23:03:39.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.005+0000 7ff4637fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff4641067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:39.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.005+0000 7ff4637fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff4641067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50300/0 (socket says 172.21.15.102:50300) 2024-04-24T23:03:39.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.005+0000 7ff4637fe700 1 -- 172.21.15.102:0/1088645509 learned_addr learned my addr 172.21.15.102:0/1088645509 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:39.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.006+0000 7ff4637fe700 1 -- 172.21.15.102:0/1088645509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4641075a0 con 0x7ff4641063a0 2024-04-24T23:03:39.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.006+0000 7ff4637fe700 1 --2- 172.21.15.102:0/1088645509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff4641067c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098a0 tx=0x7ff44c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fe2738318cc057 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:39.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.006+0000 7ff4627fc700 1 -- 172.21.15.102:0/1088645509 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c004160 con 0x7ff4641063a0 2024-04-24T23:03:39.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.006+0000 7ff4627fc700 1 -- 172.21.15.102:0/1088645509 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff44c004300 con 0x7ff4641063a0 2024-04-24T23:03:39.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.006+0000 7ff4627fc700 1 -- 172.21.15.102:0/1088645509 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c011a50 con 0x7ff4641063a0 2024-04-24T23:03:39.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.007+0000 7ff469eca700 1 -- 172.21.15.102:0/1088645509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 msgr2=0x7ff4641067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:39.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.007+0000 7ff469eca700 1 --2- 172.21.15.102:0/1088645509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff4641067c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098a0 tx=0x7ff44c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:39.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.007+0000 7ff469eca700 1 -- 172.21.15.102:0/1088645509 shutdown_connections 2024-04-24T23:03:39.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.007+0000 7ff469eca700 1 --2- 172.21.15.102:0/1088645509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff4641067c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:39.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.007+0000 7ff469eca700 1 -- 172.21.15.102:0/1088645509 >> 172.21.15.102:0/1088645509 conn(0x7ff4641019a0 msgr2=0x7ff464103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:39.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.007+0000 7ff469eca700 1 -- 172.21.15.102:0/1088645509 shutdown_connections 2024-04-24T23:03:39.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.007+0000 7ff469eca700 1 -- 172.21.15.102:0/1088645509 wait complete. 2024-04-24T23:03:39.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.007+0000 7ff469eca700 1 Processor -- start 2024-04-24T23:03:39.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.008+0000 7ff469eca700 1 -- start start 2024-04-24T23:03:39.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.008+0000 7ff469eca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff464102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:39.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.008+0000 7ff469eca700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410a950 con 0x7ff4641063a0 2024-04-24T23:03:39.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.008+0000 7ff4637fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff464102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.008+0000 7ff4637fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff464102d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50304/0 (socket says 172.21.15.102:50304) 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.008+0000 7ff4637fe700 1 -- 172.21.15.102:0/585108561 learned_addr learned my addr 172.21.15.102:0/585108561 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.008+0000 7ff4637fe700 1 -- 172.21.15.102:0/585108561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46419e950 con 0x7ff4641063a0 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.008+0000 7ff4637fe700 1 --2- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff464102d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff44c006ba0 tx=0x7ff44c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.008+0000 7ff460ff9700 1 -- 172.21.15.102:0/585108561 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c003af0 con 0x7ff4641063a0 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.009+0000 7ff460ff9700 1 -- 172.21.15.102:0/585108561 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff44c01a8e0 con 0x7ff4641063a0 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.009+0000 7ff460ff9700 1 -- 172.21.15.102:0/585108561 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c024900 con 0x7ff4641063a0 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.009+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46419eb20 con 0x7ff4641063a0 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.009+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46419eec0 con 0x7ff4641063a0 2024-04-24T23:03:39.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.010+0000 7ff460ff9700 1 -- 172.21.15.102:0/585108561 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff44c022020 con 0x7ff4641063a0 2024-04-24T23:03:39.827 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.010+0000 7ff460ff9700 1 --2- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7ff450039050 0x7ff45003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:39.827 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.010+0000 7ff460ff9700 1 -- 172.21.15.102:0/585108561 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff44c04d3d0 con 0x7ff4641063a0 2024-04-24T23:03:39.827 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.010+0000 7ff462ffd700 1 --2- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7ff450039050 0x7ff45003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:39.827 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.010+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff464063890 con 0x7ff4641063a0 2024-04-24T23:03:39.827 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.010+0000 7ff462ffd700 1 --2- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7ff450039050 0x7ff45003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff454006fd0 tx=0x7ff454006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:39.827 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.015+0000 7ff460ff9700 1 -- 172.21.15.102:0/585108561 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff44c00b3e0 con 0x7ff4641063a0 2024-04-24T23:03:39.827 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.222+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff46402dea0 con 0x7ff4641063a0 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.224+0000 7ff460ff9700 1 -- 172.21.15.102:0/585108561 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff44c020c10 con 0x7ff4641063a0 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7ff450039050 msgr2=0x7ff45003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 --2- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7ff450039050 0x7ff45003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff454006fd0 tx=0x7ff454006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 msgr2=0x7ff464102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 --2- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff464102d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff44c006ba0 tx=0x7ff44c011ad0 comp rx=0 tx=0).stop 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 shutdown_connections 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 --2- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7ff450039050 0x7ff45003b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 --2- 172.21.15.102:0/585108561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4641063a0 0x7ff464102d70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 >> 172.21.15.102:0/585108561 conn(0x7ff4641019a0 msgr2=0x7ff464103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 shutdown_connections 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:39.226+0000 7ff469eca700 1 -- 172.21.15.102:0/585108561 wait complete. 2024-04-24T23:03:39.828 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling the dashboard module... 2024-04-24T23:03:40.508 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:40 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/585108561' entity='client.admin' 2024-04-24T23:03:40.508 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:40 smithi102 ceph-mon[29216]: from='mgr.14120 172.21.15.102:0/2336077945' entity='mgr.x' 2024-04-24T23:03:41.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:41 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1508396973' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T23:03:41.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:41 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: ignoring --setuser ceph since I am not root 2024-04-24T23:03:41.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:41 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: ignoring --setgroup ceph since I am not root 2024-04-24T23:03:41.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:41 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:41.402+0000 7f8afe955200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:03:41.891 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:41 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:41.587+0000 7f8afe955200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:03:41.891 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:41 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:41.691+0000 7f8afe955200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:03:41.899 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.231+0000 7f7c9f014700 1 Processor -- start 2024-04-24T23:03:41.899 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.231+0000 7f7c9f014700 1 -- start start 2024-04-24T23:03:41.900 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.231+0000 7f7c9f014700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:41.900 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.231+0000 7f7c9f014700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98106a80 con 0x7f7c98106090 2024-04-24T23:03:41.900 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.232+0000 7f7c9cdb0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:41.900 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.232+0000 7f7c9cdb0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50306/0 (socket says 172.21.15.102:50306) 2024-04-24T23:03:41.900 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.232+0000 7f7c9cdb0700 1 -- 172.21.15.102:0/3217051820 learned_addr learned my addr 172.21.15.102:0/3217051820 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:41.900 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.232+0000 7f7c9cdb0700 1 -- 172.21.15.102:0/3217051820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c98109020 con 0x7f7c98106090 2024-04-24T23:03:41.900 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.232+0000 7f7c9cdb0700 1 --2- 172.21.15.102:0/3217051820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981064b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c940098a0 tx=0x7f7c94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef9aa0b562fbd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:41.900 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.232+0000 7f7c8f7fe700 1 -- 172.21.15.102:0/3217051820 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c94004160 con 0x7f7c98106090 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.232+0000 7f7c8f7fe700 1 -- 172.21.15.102:0/3217051820 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c94004300 con 0x7f7c98106090 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.232+0000 7f7c8f7fe700 1 -- 172.21.15.102:0/3217051820 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c94011a50 con 0x7f7c98106090 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 -- 172.21.15.102:0/3217051820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 msgr2=0x7f7c981064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 --2- 172.21.15.102:0/3217051820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981064b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c940098a0 tx=0x7f7c94009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 -- 172.21.15.102:0/3217051820 shutdown_connections 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 --2- 172.21.15.102:0/3217051820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981064b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 -- 172.21.15.102:0/3217051820 >> 172.21.15.102:0/3217051820 conn(0x7f7c981019c0 msgr2=0x7f7c98103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 -- 172.21.15.102:0/3217051820 shutdown_connections 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 -- 172.21.15.102:0/3217051820 wait complete. 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 Processor -- start 2024-04-24T23:03:41.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 -- start start 2024-04-24T23:03:41.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981088a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:41.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.233+0000 7f7c9f014700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9810c3d0 con 0x7f7c98106090 2024-04-24T23:03:41.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c9cdb0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:41.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c9cdb0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981088a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50312/0 (socket says 172.21.15.102:50312) 2024-04-24T23:03:41.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c9cdb0700 1 -- 172.21.15.102:0/1508396973 learned_addr learned my addr 172.21.15.102:0/1508396973 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:41.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c9cdb0700 1 -- 172.21.15.102:0/1508396973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c98108de0 con 0x7f7c98106090 2024-04-24T23:03:41.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c9cdb0700 1 --2- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981088a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c94006b20 tx=0x7f7c94011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:41.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c8dffb700 1 -- 172.21.15.102:0/1508396973 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c940039e0 con 0x7f7c98106090 2024-04-24T23:03:41.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c8dffb700 1 -- 172.21.15.102:0/1508396973 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c9401a8e0 con 0x7f7c98106090 2024-04-24T23:03:41.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c8dffb700 1 -- 172.21.15.102:0/1508396973 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c940247e0 con 0x7f7c98106090 2024-04-24T23:03:41.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c98106fd0 con 0x7f7c98106090 2024-04-24T23:03:41.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.234+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c98107510 con 0x7f7c98106090 2024-04-24T23:03:41.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.235+0000 7f7c8dffb700 1 -- 172.21.15.102:0/1508396973 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7c94022020 con 0x7f7c98106090 2024-04-24T23:03:41.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.235+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9810c550 con 0x7f7c98106090 2024-04-24T23:03:41.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.235+0000 7f7c8dffb700 1 --2- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7c80039050 0x7f7c8003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:41.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.235+0000 7f7c8dffb700 1 -- 172.21.15.102:0/1508396973 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c9404d460 con 0x7f7c98106090 2024-04-24T23:03:41.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.235+0000 7f7c8ffff700 1 --2- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7c80039050 0x7f7c8003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:41.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.236+0000 7f7c8ffff700 1 --2- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7c80039050 0x7f7c8003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7c88006fd0 tx=0x7f7c88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:41.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.240+0000 7f7c8dffb700 1 -- 172.21.15.102:0/1508396973 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7c9402c3a0 con 0x7f7c98106090 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:40.507+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7c98107c70 con 0x7f7c98106090 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.225+0000 7f7c8dffb700 1 -- 172.21.15.102:0/1508396973 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7c9402ad60 con 0x7f7c98106090 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.227+0000 7f7c8dffb700 1 -- 172.21.15.102:0/1508396973 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f7c94017ca0 con 0x7f7c98106090 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.228+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7c80039050 msgr2=0x7f7c8003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.228+0000 7f7c9f014700 1 --2- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7c80039050 0x7f7c8003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7c88006fd0 tx=0x7f7c88006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.228+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 msgr2=0x7f7c981088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.228+0000 7f7c9f014700 1 --2- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981088a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c94006b20 tx=0x7f7c94011ad0 comp rx=0 tx=0).stop 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.228+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 shutdown_connections 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.228+0000 7f7c9f014700 1 --2- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f7c80039050 0x7f7c8003b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.228+0000 7f7c9f014700 1 --2- 172.21.15.102:0/1508396973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c98106090 0x7f7c981088a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.228+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 >> 172.21.15.102:0/1508396973 conn(0x7f7c981019c0 msgr2=0x7f7c981024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.228+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 shutdown_connections 2024-04-24T23:03:41.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.229+0000 7f7c9f014700 1 -- 172.21.15.102:0/1508396973 wait complete. 2024-04-24T23:03:42.150 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:42 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:42.008+0000 7f8afe955200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:03:42.150 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:42 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:42.093+0000 7f8afe955200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:03:42.560 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:42 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1508396973' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T23:03:42.560 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:42 smithi102 ceph-mon[29216]: mgrmap e9: x(active, since 20s) 2024-04-24T23:03:43.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:43.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.353+0000 7fdca4284700 1 Processor -- start 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.353+0000 7fdca4284700 1 -- start start 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.354+0000 7fdca4284700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c10ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.354+0000 7fdca4284700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c07be90 con 0x7fdc9c108890 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.354+0000 7fdca2020700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c10ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.354+0000 7fdca2020700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c10ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50336/0 (socket says 172.21.15.102:50336) 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.354+0000 7fdca2020700 1 -- 172.21.15.102:0/2571194926 learned_addr learned my addr 172.21.15.102:0/2571194926 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.354+0000 7fdca2020700 1 -- 172.21.15.102:0/2571194926 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c10b1c0 con 0x7fdc9c108890 2024-04-24T23:03:43.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.354+0000 7fdca2020700 1 --2- 172.21.15.102:0/2571194926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c10ac80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc980098a0 tx=0x7fdc98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9122e3bad039eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca101e700 1 -- 172.21.15.102:0/2571194926 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc98004160 con 0x7fdc9c108890 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca101e700 1 -- 172.21.15.102:0/2571194926 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc98004300 con 0x7fdc9c108890 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca101e700 1 -- 172.21.15.102:0/2571194926 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc98011a50 con 0x7fdc9c108890 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca4284700 1 -- 172.21.15.102:0/2571194926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 msgr2=0x7fdc9c10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca4284700 1 --2- 172.21.15.102:0/2571194926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c10ac80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc980098a0 tx=0x7fdc98009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca4284700 1 -- 172.21.15.102:0/2571194926 shutdown_connections 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca4284700 1 --2- 172.21.15.102:0/2571194926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c10ac80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca4284700 1 -- 172.21.15.102:0/2571194926 >> 172.21.15.102:0/2571194926 conn(0x7fdc9c1022a0 msgr2=0x7fdc9c1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca4284700 1 -- 172.21.15.102:0/2571194926 shutdown_connections 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.355+0000 7fdca4284700 1 -- 172.21.15.102:0/2571194926 wait complete. 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.356+0000 7fdca4284700 1 Processor -- start 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.356+0000 7fdca4284700 1 -- start start 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.356+0000 7fdca4284700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.356+0000 7fdca4284700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c10c170 con 0x7fdc9c108890 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.356+0000 7fdca2020700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.356+0000 7fdca2020700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50342/0 (socket says 172.21.15.102:50342) 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.356+0000 7fdca2020700 1 -- 172.21.15.102:0/423542542 learned_addr learned my addr 172.21.15.102:0/423542542 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.356+0000 7fdca2020700 1 -- 172.21.15.102:0/423542542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c19c0d0 con 0x7fdc9c108890 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.357+0000 7fdca2020700 1 --2- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c19bb90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc98006b20 tx=0x7fdc98011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.357+0000 7fdc937fe700 1 -- 172.21.15.102:0/423542542 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc98003b70 con 0x7fdc9c108890 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.357+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9c19c3c0 con 0x7fdc9c108890 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.357+0000 7fdc937fe700 1 -- 172.21.15.102:0/423542542 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc9801a8e0 con 0x7fdc9c108890 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.357+0000 7fdc937fe700 1 -- 172.21.15.102:0/423542542 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc980249b0 con 0x7fdc9c108890 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.357+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9c19f050 con 0x7fdc9c108890 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.358+0000 7fdc937fe700 1 -- 172.21.15.102:0/423542542 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fdc98022020 con 0x7fdc9c108890 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.358+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc9c063890 con 0x7fdc9c108890 2024-04-24T23:03:43.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.358+0000 7fdc937fe700 1 --2- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fdc880390a0 0x7fdc8803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.358+0000 7fdc937fe700 1 -- 172.21.15.102:0/423542542 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc9804d890 con 0x7fdc9c108890 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.358+0000 7fdca181f700 1 -- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fdc880390a0 msgr2=0x7fdc8803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3017038970 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.358+0000 7fdca181f700 1 --2- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fdc880390a0 0x7fdc8803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.367+0000 7fdc937fe700 1 -- 172.21.15.102:0/423542542 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdc9802c3a0 con 0x7fdc9c108890 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.558+0000 7fdca181f700 1 -- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fdc880390a0 msgr2=0x7fdc8803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3017038970 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.558+0000 7fdca181f700 1 --2- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fdc880390a0 0x7fdc8803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.673+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdc9c19f760 con 0x7fdc9c108890 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.673+0000 7fdc937fe700 1 -- 172.21.15.102:0/423542542 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fdc9804d1e0 con 0x7fdc9c108890 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fdc880390a0 msgr2=0x7fdc8803b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 --2- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fdc880390a0 0x7fdc8803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 msgr2=0x7fdc9c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 --2- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c19bb90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc98006b20 tx=0x7fdc98011ad0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 shutdown_connections 2024-04-24T23:03:43.252 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 --2- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7fdc880390a0 0x7fdc8803b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.252 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 --2- 172.21.15.102:0/423542542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc9c108890 0x7fdc9c19bb90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.252 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 >> 172.21.15.102:0/423542542 conn(0x7fdc9c1022a0 msgr2=0x7fdc9c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.252 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 shutdown_connections 2024-04-24T23:03:43.252 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.675+0000 7fdca4284700 1 -- 172.21.15.102:0/423542542 wait complete. 2024-04-24T23:03:43.252 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for the mgr to restart... 2024-04-24T23:03:43.252 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr epoch 9... 2024-04-24T23:03:43.488 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:43 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/423542542' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:03:43.489 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:43 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:43.325+0000 7f8afe955200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:03:43.821 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:43 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:43.502+0000 7f8afe955200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:03:43.822 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:43 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:43.676+0000 7f8afe955200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:03:44.267 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:43 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:43.971+0000 7f8afe955200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:03:44.267 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:44 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:44.064+0000 7f8afe955200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:03:44.268 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:44 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:44.265+0000 7f8afe955200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:03:44.571 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:44 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:44.459+0000 7f8afe955200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:03:45.321 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:45 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:45.054+0000 7f8afe955200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:03:45.781 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:45 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:45.481+0000 7f8afe955200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:03:45.782 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:45 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:45.577+0000 7f8afe955200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:03:45.782 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:45 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:45.779+0000 7f8afe955200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:03:46.662 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:46 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:46.322+0000 7f8afe955200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:03:46.662 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:46 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:46.413+0000 7f8afe955200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:03:47.010 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:46 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:46.660+0000 7f8afe955200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:03:47.010 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:46 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:46.796+0000 7f8afe955200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:03:47.010 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:46 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:46.892+0000 7f8afe955200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:03:47.010 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:47 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:47.008+0000 7f8afe955200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:03:47.321 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:47 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:47.100+0000 7f8afe955200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:03:48.186 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:47 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:47.916+0000 7f8afe955200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:03:48.186 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:03:48.132+0000 7f8afe955200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: Active manager daemon x restarted 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: Activating manager daemon x 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: mgrmap e10: x(active, starting, since 0.00326054s) 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:03:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: Manager daemon x is now available 2024-04-24T23:03:48.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:03:48.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:03:48.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:03:49.975 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:49.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T23:03:49.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:03:49.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:49.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.717+0000 7f63c8ab5700 1 Processor -- start 2024-04-24T23:03:49.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.717+0000 7f63c8ab5700 1 -- start start 2024-04-24T23:03:49.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.717+0000 7f63c8ab5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4108610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.717+0000 7f63c8ab5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4108be0 con 0x7f63c41081f0 2024-04-24T23:03:49.976 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.717+0000 7f63c259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4108610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.717+0000 7f63c259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4108610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50350/0 (socket says 172.21.15.102:50350) 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.717+0000 7f63c259c700 1 -- 172.21.15.102:0/2915580324 learned_addr learned my addr 172.21.15.102:0/2915580324 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.718+0000 7f63c259c700 1 -- 172.21.15.102:0/2915580324 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c4106e50 con 0x7f63c41081f0 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.718+0000 7f63c259c700 1 --2- 172.21.15.102:0/2915580324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4108610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63b400bb40 tx=0x7f63b400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc46bf82a07cde1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.718+0000 7f63c159a700 1 -- 172.21.15.102:0/2915580324 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b400dbe0 con 0x7f63c41081f0 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.718+0000 7f63c159a700 1 -- 172.21.15.102:0/2915580324 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63b40044e0 con 0x7f63c41081f0 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.718+0000 7f63c159a700 1 -- 172.21.15.102:0/2915580324 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b4010470 con 0x7f63c41081f0 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.719+0000 7f63c8ab5700 1 -- 172.21.15.102:0/2915580324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 msgr2=0x7f63c4108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.719+0000 7f63c8ab5700 1 --2- 172.21.15.102:0/2915580324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4108610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63b400bb40 tx=0x7f63b400be50 comp rx=0 tx=0).stop 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.719+0000 7f63c8ab5700 1 -- 172.21.15.102:0/2915580324 shutdown_connections 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.719+0000 7f63c8ab5700 1 --2- 172.21.15.102:0/2915580324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4108610 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.719+0000 7f63c8ab5700 1 -- 172.21.15.102:0/2915580324 >> 172.21.15.102:0/2915580324 conn(0x7f63c4101980 msgr2=0x7f63c4103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.719+0000 7f63c8ab5700 1 -- 172.21.15.102:0/2915580324 shutdown_connections 2024-04-24T23:03:49.977 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.719+0000 7f63c8ab5700 1 -- 172.21.15.102:0/2915580324 wait complete. 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.720+0000 7f63c8ab5700 1 Processor -- start 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.720+0000 7f63c8ab5700 1 -- start start 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.720+0000 7f63c8ab5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.720+0000 7f63c8ab5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4105d40 con 0x7f63c41081f0 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.720+0000 7f63c259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.720+0000 7f63c259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50362/0 (socket says 172.21.15.102:50362) 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.720+0000 7f63c259c700 1 -- 172.21.15.102:0/3891846203 learned_addr learned my addr 172.21.15.102:0/3891846203 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.721+0000 7f63c259c700 1 -- 172.21.15.102:0/3891846203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c4199f40 con 0x7f63c41081f0 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.721+0000 7f63c259c700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4199a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63b4003bf0 tx=0x7f63b4003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.721+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b4027070 con 0x7f63c41081f0 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.721+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c419a1d0 con 0x7f63c41081f0 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.721+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63b4010bd0 con 0x7f63c41081f0 2024-04-24T23:03:49.978 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.721+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b4022b30 con 0x7f63c41081f0 2024-04-24T23:03:49.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.721+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c419a630 con 0x7f63c41081f0 2024-04-24T23:03:49.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f63b402a070 con 0x7f63c41081f0 2024-04-24T23:03:49.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7f63b37fe700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 0x7f63ac03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 --> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f63c4063890 con 0x7f63ac0390a0 2024-04-24T23:03:49.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f63b401f070 con 0x7f63c41081f0 2024-04-24T23:03:49.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7f63c1d9b700 1 -- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 msgr2=0x7f63ac03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3017038970 2024-04-24T23:03:49.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7f63c1d9b700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 0x7f63ac03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:03:49.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.923+0000 7f63c1d9b700 1 -- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 msgr2=0x7f63ac03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3017038970 2024-04-24T23:03:49.979 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.923+0000 7f63c1d9b700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 0x7f63ac03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:44.324+0000 7f63c1d9b700 1 -- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 msgr2=0x7f63ac03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3017038970 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:44.324+0000 7f63c1d9b700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 0x7f63ac03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:45.125+0000 7f63c1d9b700 1 -- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 msgr2=0x7f63ac03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3017038970 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:45.125+0000 7f63c1d9b700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 0x7f63ac03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.726+0000 7f63c1d9b700 1 -- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 msgr2=0x7f63ac03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3017038970 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.726+0000 7f63c1d9b700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 0x7f63ac03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.135+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f63b4022410 con 0x7f63c41081f0 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.135+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 msgr2=0x7f63ac03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.135+0000 7f63b37fe700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 0x7f63ac03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.980 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.139+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f63b40519c0 con 0x7f63c41081f0 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.139+0000 7f63b37fe700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f63ac03c9c0 0x7f63ac03edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.140+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f63c4063890 con 0x7f63ac03c9c0 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.140+0000 7f63c1d9b700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f63ac03c9c0 0x7f63ac03edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.140+0000 7f63c1d9b700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f63ac03c9c0 0x7f63ac03edb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f63b8003a10 tx=0x7f63b80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.141+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f63c4063890 con 0x7f63ac03c9c0 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.147+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f63c41917c0 con 0x7f63ac03c9c0 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63b37fe700 1 -- 172.21.15.102:0/3891846203 <== mgr.14152 v2:172.21.15.102:6800/3611055373 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f63c41917c0 con 0x7f63ac03c9c0 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f63ac03c9c0 msgr2=0x7f63ac03edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f63ac03c9c0 0x7f63ac03edb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f63b8003a10 tx=0x7f63b80092b0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 msgr2=0x7f63c4199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.981 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4199a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63b4003bf0 tx=0x7f63b4003fb0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 shutdown_connections 2024-04-24T23:03:49.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f63ac03c9c0 0x7f63ac03edb0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c41081f0 0x7f63c4199a00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 --2- 172.21.15.102:0/3891846203 >> [v2:172.21.15.102:6800/3017038970,v1:172.21.15.102:6801/3017038970] conn(0x7f63ac0390a0 0x7f63ac03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.982 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 >> 172.21.15.102:0/3891846203 conn(0x7f63c4101980 msgr2=0x7f63c4190020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:49.983 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 shutdown_connections 2024-04-24T23:03:49.983 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:49.148+0000 7f63c8ab5700 1 -- 172.21.15.102:0/3891846203 wait complete. 2024-04-24T23:03:49.983 INFO:teuthology.orchestra.run.smithi102.stdout:mgr epoch 9 is available 2024-04-24T23:03:49.983 INFO:teuthology.orchestra.run.smithi102.stdout:Generating a dashboard self-signed certificate... 2024-04-24T23:03:50.245 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:50 smithi102 ceph-mon[29216]: [24/Apr/2024:23:03:48] ENGINE Bus STARTING 2024-04-24T23:03:50.245 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:50 smithi102 ceph-mon[29216]: [24/Apr/2024:23:03:48] ENGINE Serving on http://172.21.15.102:8765 2024-04-24T23:03:50.245 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:50 smithi102 ceph-mon[29216]: [24/Apr/2024:23:03:48] ENGINE Serving on https://172.21.15.102:7150 2024-04-24T23:03:50.245 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:50 smithi102 ceph-mon[29216]: [24/Apr/2024:23:03:48] ENGINE Bus STARTED 2024-04-24T23:03:50.245 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:50 smithi102 ceph-mon[29216]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:03:50.245 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:50 smithi102 ceph-mon[29216]: mgrmap e11: x(active, since 1.00815s) 2024-04-24T23:03:50.245 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:50 smithi102 ceph-mon[29216]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:03:50.245 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.451+0000 7f1920256700 1 Processor -- start 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.452+0000 7f1920256700 1 -- start start 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.452+0000 7f1920256700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f1918106510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.452+0000 7f1920256700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1918106ae0 con 0x7f19181060f0 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.452+0000 7f191dff2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f1918106510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.452+0000 7f191dff2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f1918106510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50612/0 (socket says 172.21.15.102:50612) 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.452+0000 7f191dff2700 1 -- 172.21.15.102:0/3233384044 learned_addr learned my addr 172.21.15.102:0/3233384044 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.452+0000 7f191dff2700 1 -- 172.21.15.102:0/3233384044 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1918109080 con 0x7f19181060f0 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.452+0000 7f191dff2700 1 --2- 172.21.15.102:0/3233384044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f1918106510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f19140098a0 tx=0x7f1914009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733a4899220105aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:51.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f191cff0700 1 -- 172.21.15.102:0/3233384044 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1914004160 con 0x7f19181060f0 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f191cff0700 1 -- 172.21.15.102:0/3233384044 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1914004300 con 0x7f19181060f0 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f191cff0700 1 -- 172.21.15.102:0/3233384044 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1914011a50 con 0x7f19181060f0 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f1920256700 1 -- 172.21.15.102:0/3233384044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 msgr2=0x7f1918106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f1920256700 1 --2- 172.21.15.102:0/3233384044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f1918106510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f19140098a0 tx=0x7f1914009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f1920256700 1 -- 172.21.15.102:0/3233384044 shutdown_connections 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f1920256700 1 --2- 172.21.15.102:0/3233384044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f1918106510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f1920256700 1 -- 172.21.15.102:0/3233384044 >> 172.21.15.102:0/3233384044 conn(0x7f1918101a00 msgr2=0x7f1918103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f1920256700 1 -- 172.21.15.102:0/3233384044 shutdown_connections 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.453+0000 7f1920256700 1 -- 172.21.15.102:0/3233384044 wait complete. 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.454+0000 7f1920256700 1 Processor -- start 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.454+0000 7f1920256700 1 -- start start 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.454+0000 7f1920256700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f191819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.454+0000 7f1920256700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f191810c430 con 0x7f19181060f0 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.454+0000 7f191dff2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f191819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.454+0000 7f191dff2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f191819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50618/0 (socket says 172.21.15.102:50618) 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.454+0000 7f191dff2700 1 -- 172.21.15.102:0/2844295298 learned_addr learned my addr 172.21.15.102:0/2844295298 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.455+0000 7f191dff2700 1 -- 172.21.15.102:0/2844295298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191819c1d0 con 0x7f19181060f0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.455+0000 7f191dff2700 1 --2- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f191819bc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1914011a20 tx=0x7f1914011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.455+0000 7f190effd700 1 -- 172.21.15.102:0/2844295298 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1914003af0 con 0x7f19181060f0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.455+0000 7f190effd700 1 -- 172.21.15.102:0/2844295298 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f191401a8e0 con 0x7f19181060f0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.455+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191819c460 con 0x7f19181060f0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.455+0000 7f190effd700 1 -- 172.21.15.102:0/2844295298 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1914024860 con 0x7f19181060f0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.455+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191819f150 con 0x7f19181060f0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.456+0000 7f190effd700 1 -- 172.21.15.102:0/2844295298 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f1914022020 con 0x7f19181060f0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.456+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1918063890 con 0x7f19181060f0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.456+0000 7f190effd700 1 --2- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1904038ff0 0x7f190403b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.456+0000 7f190effd700 1 -- 172.21.15.102:0/2844295298 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f191404d250 con 0x7f19181060f0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.456+0000 7f191d7f1700 1 --2- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1904038ff0 0x7f190403b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:51.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.457+0000 7f191d7f1700 1 --2- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1904038ff0 0x7f190403b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1908006fd0 tx=0x7f1908006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.461+0000 7f190effd700 1 -- 172.21.15.102:0/2844295298 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f191400f2e0 con 0x7f19181060f0 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.680+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f1918102c00 con 0x7f1904038ff0 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.811+0000 7f190effd700 1 -- 172.21.15.102:0/2844295298 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f1918102c00 con 0x7f1904038ff0 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.813+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1904038ff0 msgr2=0x7f190403b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.813+0000 7f1920256700 1 --2- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1904038ff0 0x7f190403b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1908006fd0 tx=0x7f1908006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.813+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 msgr2=0x7f191819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.813+0000 7f1920256700 1 --2- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f191819bc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1914011a20 tx=0x7f1914011b00 comp rx=0 tx=0).stop 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.813+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 shutdown_connections 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.813+0000 7f1920256700 1 --2- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1904038ff0 0x7f190403b4b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.813+0000 7f1920256700 1 --2- 172.21.15.102:0/2844295298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19181060f0 0x7f191819bc90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.813+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 >> 172.21.15.102:0/2844295298 conn(0x7f1918101a00 msgr2=0x7f19181024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.814+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 shutdown_connections 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:50.814+0000 7f1920256700 1 -- 172.21.15.102:0/2844295298 wait complete. 2024-04-24T23:03:51.466 INFO:teuthology.orchestra.run.smithi102.stdout:Creating initial admin user... 2024-04-24T23:03:52.034 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:51 smithi102 ceph-mon[29216]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:52.034 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:03:52.034 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:03:52.034 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:51 smithi102 ceph-mon[29216]: mgrmap e12: x(active, since 2s) 2024-04-24T23:03:52.034 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:03:53.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:52 smithi102 ceph-mon[29216]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:03:53.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:52 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:03:53.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ztFpSpaIx7dSvRuADUuWGuBJX2d4sBhK8cVpzrAHIkReIeqEB0Ls.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713999832, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T23:03:53.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.876+0000 7f874fbc9700 1 Processor -- start 2024-04-24T23:03:53.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.876+0000 7f874fbc9700 1 -- start start 2024-04-24T23:03:53.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874fbc9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f8748108640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874fbc9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8748108c10 con 0x7f8748108220 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874d965700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f8748108640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874d965700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f8748108640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50628/0 (socket says 172.21.15.102:50628) 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874d965700 1 -- 172.21.15.102:0/1751062745 learned_addr learned my addr 172.21.15.102:0/1751062745 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874d965700 1 -- 172.21.15.102:0/1751062745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8748106e80 con 0x7f8748108220 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874d965700 1 --2- 172.21.15.102:0/1751062745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f8748108640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87380098a0 tx=0x7f8738009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211193838d9193bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874c963700 1 -- 172.21.15.102:0/1751062745 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8738004160 con 0x7f8748108220 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874c963700 1 -- 172.21.15.102:0/1751062745 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8738004300 con 0x7f8748108220 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.877+0000 7f874c963700 1 -- 172.21.15.102:0/1751062745 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8738011a50 con 0x7f8748108220 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.878+0000 7f874fbc9700 1 -- 172.21.15.102:0/1751062745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 msgr2=0x7f8748108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.878+0000 7f874fbc9700 1 --2- 172.21.15.102:0/1751062745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f8748108640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87380098a0 tx=0x7f8738009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.878+0000 7f874fbc9700 1 -- 172.21.15.102:0/1751062745 shutdown_connections 2024-04-24T23:03:53.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.878+0000 7f874fbc9700 1 --2- 172.21.15.102:0/1751062745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f8748108640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.878+0000 7f874fbc9700 1 -- 172.21.15.102:0/1751062745 >> 172.21.15.102:0/1751062745 conn(0x7f8748101970 msgr2=0x7f8748103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.878+0000 7f874fbc9700 1 -- 172.21.15.102:0/1751062745 shutdown_connections 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.878+0000 7f874fbc9700 1 -- 172.21.15.102:0/1751062745 wait complete. 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.878+0000 7f874fbc9700 1 Processor -- start 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874fbc9700 1 -- start start 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874fbc9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f874819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874fbc9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f874810b240 con 0x7f8748108220 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874d965700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f874819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874d965700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f874819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50640/0 (socket says 172.21.15.102:50640) 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874d965700 1 -- 172.21.15.102:0/4002563751 learned_addr learned my addr 172.21.15.102:0/4002563751 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874d965700 1 -- 172.21.15.102:0/4002563751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874819b040 con 0x7f8748108220 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874d965700 1 --2- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f874819ab00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8738011a00 tx=0x7f8738011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f873effd700 1 -- 172.21.15.102:0/4002563751 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8738003af0 con 0x7f8748108220 2024-04-24T23:03:53.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f873effd700 1 -- 172.21.15.102:0/4002563751 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f873801a8e0 con 0x7f8748108220 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874819b2d0 con 0x7f8748108220 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f873effd700 1 -- 172.21.15.102:0/4002563751 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8738024840 con 0x7f8748108220 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.879+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874819dfc0 con 0x7f8748108220 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.880+0000 7f873effd700 1 -- 172.21.15.102:0/4002563751 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f8738022020 con 0x7f8748108220 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.880+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f874810b3c0 con 0x7f8748108220 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.880+0000 7f873effd700 1 --2- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8734039100 0x7f873403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.880+0000 7f873effd700 1 -- 172.21.15.102:0/4002563751 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f873804d640 con 0x7f8748108220 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.881+0000 7f874d164700 1 --2- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8734039100 0x7f873403b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.881+0000 7f874d164700 1 --2- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8734039100 0x7f873403b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8744006fd0 tx=0x7f8744006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.886+0000 7f873effd700 1 -- 172.21.15.102:0/4002563751 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f87380172b0 con 0x7f8748108220 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.106+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f8748063890 con 0x7f8734039100 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.331+0000 7f873effd700 1 -- 172.21.15.102:0/4002563751 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f8748063890 con 0x7f8734039100 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.333+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8734039100 msgr2=0x7f873403b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.084 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.333+0000 7f874fbc9700 1 --2- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8734039100 0x7f873403b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8744006fd0 tx=0x7f8744006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.333+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 msgr2=0x7f874819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.333+0000 7f874fbc9700 1 --2- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f874819ab00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8738011a00 tx=0x7f8738011ae0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.333+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 shutdown_connections 2024-04-24T23:03:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.333+0000 7f874fbc9700 1 --2- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8734039100 0x7f873403b5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.333+0000 7f874fbc9700 1 --2- 172.21.15.102:0/4002563751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8748108220 0x7f874819ab00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.333+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 >> 172.21.15.102:0/4002563751 conn(0x7f8748101970 msgr2=0x7f8748102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.334+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 shutdown_connections 2024-04-24T23:03:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:52.334+0000 7f874fbc9700 1 -- 172.21.15.102:0/4002563751 wait complete. 2024-04-24T23:03:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:Fetching dashboard port number... 2024-04-24T23:03:54.022 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:53 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2823694029' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T23:03:54.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T23:03:54.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.548+0000 7f47750ee700 1 Processor -- start 2024-04-24T23:03:54.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.548+0000 7f47750ee700 1 -- start start 2024-04-24T23:03:54.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.548+0000 7f47750ee700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f4770106830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:54.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.548+0000 7f47750ee700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4770106e00 con 0x7f4770106410 2024-04-24T23:03:54.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.548+0000 7f476ed9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f4770106830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:54.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.549+0000 7f476ed9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f4770106830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50648/0 (socket says 172.21.15.102:50648) 2024-04-24T23:03:54.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.549+0000 7f476ed9d700 1 -- 172.21.15.102:0/2974912955 learned_addr learned my addr 172.21.15.102:0/2974912955 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:54.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.549+0000 7f476ed9d700 1 -- 172.21.15.102:0/2974912955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4770107610 con 0x7f4770106410 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.549+0000 7f476ed9d700 1 --2- 172.21.15.102:0/2974912955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f4770106830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f476000aa40 tx=0x7f476000ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296ab56993ae1ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.549+0000 7f476dd9b700 1 -- 172.21.15.102:0/2974912955 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476000b040 con 0x7f4770106410 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.549+0000 7f476dd9b700 1 -- 172.21.15.102:0/2974912955 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f476000c8f0 con 0x7f4770106410 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.549+0000 7f476dd9b700 1 -- 172.21.15.102:0/2974912955 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760003ae0 con 0x7f4770106410 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.550+0000 7f47750ee700 1 -- 172.21.15.102:0/2974912955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 msgr2=0x7f4770106830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.550+0000 7f47750ee700 1 --2- 172.21.15.102:0/2974912955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f4770106830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f476000aa40 tx=0x7f476000ad50 comp rx=0 tx=0).stop 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.550+0000 7f47750ee700 1 -- 172.21.15.102:0/2974912955 shutdown_connections 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.550+0000 7f47750ee700 1 --2- 172.21.15.102:0/2974912955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f4770106830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.550+0000 7f47750ee700 1 -- 172.21.15.102:0/2974912955 >> 172.21.15.102:0/2974912955 conn(0x7f4770101970 msgr2=0x7f4770103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.550+0000 7f47750ee700 1 -- 172.21.15.102:0/2974912955 shutdown_connections 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.550+0000 7f47750ee700 1 -- 172.21.15.102:0/2974912955 wait complete. 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.550+0000 7f47750ee700 1 Processor -- start 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f47750ee700 1 -- start start 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f47750ee700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f477019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:54.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f47750ee700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4770107d80 con 0x7f4770106410 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f476ed9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f477019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f476ed9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f477019bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50654/0 (socket says 172.21.15.102:50654) 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f476ed9d700 1 -- 172.21.15.102:0/2823694029 learned_addr learned my addr 172.21.15.102:0/2823694029 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f476ed9d700 1 -- 172.21.15.102:0/2823694029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477019c240 con 0x7f4770106410 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f476ed9d700 1 --2- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f477019bd00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4760003fc0 tx=0x7f47600040a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f4757fff700 1 -- 172.21.15.102:0/2823694029 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760025070 con 0x7f4770106410 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f4757fff700 1 -- 172.21.15.102:0/2823694029 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f476000cd60 con 0x7f4770106410 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477019c4d0 con 0x7f4770106410 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f4757fff700 1 -- 172.21.15.102:0/2823694029 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760010610 con 0x7f4770106410 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.551+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477019f1b0 con 0x7f4770106410 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.552+0000 7f4757fff700 1 -- 172.21.15.102:0/2823694029 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4760028070 con 0x7f4770106410 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.552+0000 7f4757fff700 1 --2- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f47580390a0 0x7f475803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.552+0000 7f4757fff700 1 -- 172.21.15.102:0/2823694029 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f476004dda0 con 0x7f4770106410 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.553+0000 7f476e59c700 1 --2- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f47580390a0 0x7f475803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:54.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.553+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4770063890 con 0x7f4770106410 2024-04-24T23:03:54.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.553+0000 7f476e59c700 1 --2- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f47580390a0 0x7f475803b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4764006fd0 tx=0x7f4764006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:54.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.558+0000 7f4757fff700 1 -- 172.21.15.102:0/2823694029 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4760014070 con 0x7f4770106410 2024-04-24T23:03:54.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.760+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f477002dc00 con 0x7f4770106410 2024-04-24T23:03:54.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.760+0000 7f4757fff700 1 -- 172.21.15.102:0/2823694029 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f4760017560 con 0x7f4770106410 2024-04-24T23:03:54.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.762+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f47580390a0 msgr2=0x7f475803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:54.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.762+0000 7f47750ee700 1 --2- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f47580390a0 0x7f475803b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4764006fd0 tx=0x7f4764006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:54.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.762+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 msgr2=0x7f477019bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:54.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.762+0000 7f47750ee700 1 --2- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f477019bd00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4760003fc0 tx=0x7f47600040a0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.762+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 shutdown_connections 2024-04-24T23:03:54.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.762+0000 7f47750ee700 1 --2- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f47580390a0 0x7f475803b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.762+0000 7f47750ee700 1 --2- 172.21.15.102:0/2823694029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4770106410 0x7f477019bd00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:54.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.763+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 >> 172.21.15.102:0/2823694029 conn(0x7f4770101970 msgr2=0x7f4770192860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:54.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.763+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 shutdown_connections 2024-04-24T23:03:54.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:53.763+0000 7f47750ee700 1 -- 172.21.15.102:0/2823694029 wait complete. 2024-04-24T23:03:54.700 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:03:54.700 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-04-24T23:03:54.708 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:03:54.708 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-04-24T23:03:54.708 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-04-24T23:03:54.708 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T23:03:54.710 INFO:teuthology.orchestra.run.smithi102.stdout:Ceph Dashboard is now available at: 2024-04-24T23:03:54.710 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:54.710 INFO:teuthology.orchestra.run.smithi102.stdout: URL: https://smithi102.front.sepia.ceph.com:8443/ 2024-04-24T23:03:54.710 INFO:teuthology.orchestra.run.smithi102.stdout: User: admin 2024-04-24T23:03:54.711 INFO:teuthology.orchestra.run.smithi102.stdout: Password: osr8rfcd4v 2024-04-24T23:03:54.711 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:54.711 INFO:teuthology.orchestra.run.smithi102.stdout:Saving cluster configuration to /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config directory 2024-04-24T23:03:54.712 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling autotune for osd_memory_target 2024-04-24T23:03:55.923 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:55 smithi102 ceph-mon[29216]: mgrmap e13: x(active, since 6s) 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.115+0000 7f07c1413700 1 Processor -- start 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.115+0000 7f07c1413700 1 -- start start 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.115+0000 7f07c1413700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc10ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.115+0000 7f07c1413700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc07beb0 con 0x7f07bc108880 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.115+0000 7f07baffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc10ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.115+0000 7f07baffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc10ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50668/0 (socket says 172.21.15.102:50668) 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.116+0000 7f07baffd700 1 -- 172.21.15.102:0/964865137 learned_addr learned my addr 172.21.15.102:0/964865137 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.116+0000 7f07baffd700 1 -- 172.21.15.102:0/964865137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc10b1b0 con 0x7f07bc108880 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.116+0000 7f07baffd700 1 --2- 172.21.15.102:0/964865137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc10ac70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0098a0 tx=0x7f07ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e894bf215c7e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.116+0000 7f07b9ffb700 1 -- 172.21.15.102:0/964865137 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ac004160 con 0x7f07bc108880 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.116+0000 7f07b9ffb700 1 -- 172.21.15.102:0/964865137 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07ac004300 con 0x7f07bc108880 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.116+0000 7f07b9ffb700 1 -- 172.21.15.102:0/964865137 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ac011a50 con 0x7f07bc108880 2024-04-24T23:03:55.932 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.117+0000 7f07c1413700 1 -- 172.21.15.102:0/964865137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 msgr2=0x7f07bc10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.117+0000 7f07c1413700 1 --2- 172.21.15.102:0/964865137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc10ac70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0098a0 tx=0x7f07ac009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.117+0000 7f07c1413700 1 -- 172.21.15.102:0/964865137 shutdown_connections 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.117+0000 7f07c1413700 1 --2- 172.21.15.102:0/964865137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc10ac70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.117+0000 7f07c1413700 1 -- 172.21.15.102:0/964865137 >> 172.21.15.102:0/964865137 conn(0x7f07bc1022b0 msgr2=0x7f07bc1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.117+0000 7f07c1413700 1 -- 172.21.15.102:0/964865137 shutdown_connections 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.117+0000 7f07c1413700 1 -- 172.21.15.102:0/964865137 wait complete. 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.117+0000 7f07c1413700 1 Processor -- start 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.117+0000 7f07c1413700 1 -- start start 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07c1413700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.933 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07c1413700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc10c1c0 con 0x7f07bc108880 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07baffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07baffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc19bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50680/0 (socket says 172.21.15.102:50680) 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07baffd700 1 -- 172.21.15.102:0/4139851827 learned_addr learned my addr 172.21.15.102:0/4139851827 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07baffd700 1 -- 172.21.15.102:0/4139851827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc19c1a0 con 0x7f07bc108880 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07baffd700 1 --2- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc19bc60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07ac011a00 tx=0x7f07ac011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07a3fff700 1 -- 172.21.15.102:0/4139851827 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ac003af0 con 0x7f07bc108880 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07a3fff700 1 -- 172.21.15.102:0/4139851827 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07ac01a8e0 con 0x7f07bc108880 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc19c430 con 0x7f07bc108880 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07a3fff700 1 -- 172.21.15.102:0/4139851827 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ac024840 con 0x7f07bc108880 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.118+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc19f120 con 0x7f07bc108880 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.119+0000 7f07a3fff700 1 -- 172.21.15.102:0/4139851827 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f07ac022020 con 0x7f07bc108880 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.119+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07bc063890 con 0x7f07bc108880 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.119+0000 7f07a3fff700 1 --2- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07a4039140 0x7f07a403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.934 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.119+0000 7f07a3fff700 1 -- 172.21.15.102:0/4139851827 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f07ac04d620 con 0x7f07bc108880 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.120+0000 7f07ba7fc700 1 --2- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07a4039140 0x7f07a403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.120+0000 7f07ba7fc700 1 --2- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07a4039140 0x7f07a403b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07b0006fd0 tx=0x7f07b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.124+0000 7f07a3fff700 1 -- 172.21.15.102:0/4139851827 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f07ac019c70 con 0x7f07bc108880 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.319+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f07bc19f740 con 0x7f07bc108880 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.320+0000 7f07a3fff700 1 -- 172.21.15.102:0/4139851827 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f07ac016df0 con 0x7f07bc108880 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07a4039140 msgr2=0x7f07a403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 --2- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07a4039140 0x7f07a403b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07b0006fd0 tx=0x7f07b0006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 msgr2=0x7f07bc19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 --2- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc19bc60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07ac011a00 tx=0x7f07ac011ae0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 shutdown_connections 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 --2- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07a4039140 0x7f07a403b600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 --2- 172.21.15.102:0/4139851827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07bc108880 0x7f07bc19bc60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.935 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 >> 172.21.15.102:0/4139851827 conn(0x7f07bc1022b0 msgr2=0x7f07bc102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:55.936 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 shutdown_connections 2024-04-24T23:03:55.936 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:55.322+0000 7f07c1413700 1 -- 172.21.15.102:0/4139851827 wait complete. 2024-04-24T23:03:57.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.373+0000 7f8a15d08700 1 Processor -- start 2024-04-24T23:03:57.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.373+0000 7f8a15d08700 1 -- start start 2024-04-24T23:03:57.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a15d08700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1010ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a15d08700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1007beb0 con 0x7f8a10108880 2024-04-24T23:03:57.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a0f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1010ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a0f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1010ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50694/0 (socket says 172.21.15.102:50694) 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a0f7fe700 1 -- 172.21.15.102:0/3209497400 learned_addr learned my addr 172.21.15.102:0/3209497400 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a0f7fe700 1 -- 172.21.15.102:0/3209497400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1010b1b0 con 0x7f8a10108880 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a0f7fe700 1 --2- 172.21.15.102:0/3209497400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1010ac70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a000098a0 tx=0x7f8a00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84b9b0a5633a8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a0effd700 1 -- 172.21.15.102:0/3209497400 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a00004160 con 0x7f8a10108880 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a0effd700 1 -- 172.21.15.102:0/3209497400 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a00004300 con 0x7f8a10108880 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.374+0000 7f8a0effd700 1 -- 172.21.15.102:0/3209497400 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a00011a50 con 0x7f8a10108880 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.375+0000 7f8a15d08700 1 -- 172.21.15.102:0/3209497400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 msgr2=0x7f8a1010ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.375+0000 7f8a15d08700 1 --2- 172.21.15.102:0/3209497400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1010ac70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a000098a0 tx=0x7f8a00009bb0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.375+0000 7f8a15d08700 1 -- 172.21.15.102:0/3209497400 shutdown_connections 2024-04-24T23:03:57.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.375+0000 7f8a15d08700 1 --2- 172.21.15.102:0/3209497400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1010ac70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.375+0000 7f8a15d08700 1 -- 172.21.15.102:0/3209497400 >> 172.21.15.102:0/3209497400 conn(0x7f8a101022b0 msgr2=0x7f8a101046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:57.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.375+0000 7f8a15d08700 1 -- 172.21.15.102:0/3209497400 shutdown_connections 2024-04-24T23:03:57.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.375+0000 7f8a15d08700 1 -- 172.21.15.102:0/3209497400 wait complete. 2024-04-24T23:03:57.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.376+0000 7f8a15d08700 1 Processor -- start 2024-04-24T23:03:57.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.376+0000 7f8a15d08700 1 -- start start 2024-04-24T23:03:57.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.376+0000 7f8a15d08700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1019bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.376+0000 7f8a15d08700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1010c1c0 con 0x7f8a10108880 2024-04-24T23:03:57.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.376+0000 7f8a0f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1019bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.376+0000 7f8a0f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1019bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50698/0 (socket says 172.21.15.102:50698) 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.376+0000 7f8a0f7fe700 1 -- 172.21.15.102:0/2026918705 learned_addr learned my addr 172.21.15.102:0/2026918705 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.376+0000 7f8a0f7fe700 1 -- 172.21.15.102:0/2026918705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1019c1a0 con 0x7f8a10108880 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.377+0000 7f8a0f7fe700 1 --2- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1019bc60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a00011a20 tx=0x7f8a00011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.377+0000 7f8a0d7fa700 1 -- 172.21.15.102:0/2026918705 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a000039c0 con 0x7f8a10108880 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.377+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1019c430 con 0x7f8a10108880 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.377+0000 7f8a0d7fa700 1 -- 172.21.15.102:0/2026918705 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a00003b60 con 0x7f8a10108880 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.377+0000 7f8a0d7fa700 1 -- 172.21.15.102:0/2026918705 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0001a6b0 con 0x7f8a10108880 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.377+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1019f120 con 0x7f8a10108880 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.378+0000 7f8a0d7fa700 1 -- 172.21.15.102:0/2026918705 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8a0001a9a0 con 0x7f8a10108880 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.378+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a10063890 con 0x7f8a10108880 2024-04-24T23:03:57.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.378+0000 7f8a0d7fa700 1 --2- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f89fc039150 0x7f89fc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.378+0000 7f8a0d7fa700 1 -- 172.21.15.102:0/2026918705 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8a0004d8a0 con 0x7f8a10108880 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.378+0000 7f8a06ffd700 1 --2- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f89fc039150 0x7f89fc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.378+0000 7f8a06ffd700 1 --2- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f89fc039150 0x7f89fc03b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006fd0 tx=0x7f89f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.383+0000 7f8a0d7fa700 1 -- 172.21.15.102:0/2026918705 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8a0000fdd0 con 0x7f8a10108880 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.692+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8a1019f6d0 con 0x7f8a10108880 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.694+0000 7f8a0d7fa700 1 -- 172.21.15.102:0/2026918705 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f8a00024a70 con 0x7f8a10108880 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f89fc039150 msgr2=0x7f89fc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 --2- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f89fc039150 0x7f89fc03b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006fd0 tx=0x7f89f8006e40 comp rx=0 tx=0).stop 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 msgr2=0x7f8a1019bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 --2- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1019bc60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a00011a20 tx=0x7f8a00011b00 comp rx=0 tx=0).stop 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 shutdown_connections 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 --2- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f89fc039150 0x7f89fc03b610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 --2- 172.21.15.102:0/2026918705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a10108880 0x7f8a1019bc60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 >> 172.21.15.102:0/2026918705 conn(0x7f8a101022b0 msgr2=0x7f8a10102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 shutdown_connections 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:56.696+0000 7f8a15d08700 1 -- 172.21.15.102:0/2026918705 wait complete. 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T23:03:57.288 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout: ceph telemetry on 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout:For more information see: 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:03:57.289 INFO:teuthology.orchestra.run.smithi102.stdout:Bootstrap complete. 2024-04-24T23:03:57.319 INFO:tasks.cephadm:Fetching config... 2024-04-24T23:03:57.319 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:03:57.319 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T23:03:57.336 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T23:03:57.336 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:03:57.336 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T23:03:57.391 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T23:03:57.391 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:03:57.392 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T23:03:57.455 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T23:03:57.455 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:03:57.455 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T23:03:57.510 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T23:03:57.510 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4/oogTrwYQqZGuFhFGaoC/8pV3KBalx4+pwkGkE+r3kww0yOp49llyx2ROboMyRlwufZsWIy2cEG0Z7B+D3DE3st3LMdC9jWSv6BQzVU5T9bJgWECrqLP51VfWvMy/crzor3tbNY0wLx2f77u+ACTsELzKVOWcv3EEvO3jaZissFQ3Pk2L2K4zPv/LV8eObNs6v8q5NBEhfSqoboF3jD7rJQd4IpNT8CnsmaifgjSm1sLkFjsDs0vcYBamtQ4bQg1uGUA0RZ8Hj2fe63kwnhXaM069qn5LkbhQA9QfLycnRxBunUGq1OTSVZJYW2i4RDVvlI5pQjx+GF4HsQ4ElNc2WlWjRnEMHkEQr/z0+yjjCWoT0UuQgx5c+rRjqZ59vv8UKas9eyjkRspIt+yxkZ3hVi+7lbofi3srNPE72ODkuM4eokQQ5r81TQAU7sZ7eky8E2rOMvHCR6No44z0gfyBA4ObuTGXr0jIPkLVOC3ZHGqLHq4nJbSVvrWsjvvw0= ceph-97eecd66-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:03:57.584 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4/oogTrwYQqZGuFhFGaoC/8pV3KBalx4+pwkGkE+r3kww0yOp49llyx2ROboMyRlwufZsWIy2cEG0Z7B+D3DE3st3LMdC9jWSv6BQzVU5T9bJgWECrqLP51VfWvMy/crzor3tbNY0wLx2f77u+ACTsELzKVOWcv3EEvO3jaZissFQ3Pk2L2K4zPv/LV8eObNs6v8q5NBEhfSqoboF3jD7rJQd4IpNT8CnsmaifgjSm1sLkFjsDs0vcYBamtQ4bQg1uGUA0RZ8Hj2fe63kwnhXaM069qn5LkbhQA9QfLycnRxBunUGq1OTSVZJYW2i4RDVvlI5pQjx+GF4HsQ4ElNc2WlWjRnEMHkEQr/z0+yjjCWoT0UuQgx5c+rRjqZ59vv8UKas9eyjkRspIt+yxkZ3hVi+7lbofi3srNPE72ODkuM4eokQQ5r81TQAU7sZ7eky8E2rOMvHCR6No44z0gfyBA4ObuTGXr0jIPkLVOC3ZHGqLHq4nJbSVvrWsjvvw0= ceph-97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:57.593 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4/oogTrwYQqZGuFhFGaoC/8pV3KBalx4+pwkGkE+r3kww0yOp49llyx2ROboMyRlwufZsWIy2cEG0Z7B+D3DE3st3LMdC9jWSv6BQzVU5T9bJgWECrqLP51VfWvMy/crzor3tbNY0wLx2f77u+ACTsELzKVOWcv3EEvO3jaZissFQ3Pk2L2K4zPv/LV8eObNs6v8q5NBEhfSqoboF3jD7rJQd4IpNT8CnsmaifgjSm1sLkFjsDs0vcYBamtQ4bQg1uGUA0RZ8Hj2fe63kwnhXaM069qn5LkbhQA9QfLycnRxBunUGq1OTSVZJYW2i4RDVvlI5pQjx+GF4HsQ4ElNc2WlWjRnEMHkEQr/z0+yjjCWoT0UuQgx5c+rRjqZ59vv8UKas9eyjkRspIt+yxkZ3hVi+7lbofi3srNPE72ODkuM4eokQQ5r81TQAU7sZ7eky8E2rOMvHCR6No44z0gfyBA4ObuTGXr0jIPkLVOC3ZHGqLHq4nJbSVvrWsjvvw0= ceph-97eecd66-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:03:57.634 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4/oogTrwYQqZGuFhFGaoC/8pV3KBalx4+pwkGkE+r3kww0yOp49llyx2ROboMyRlwufZsWIy2cEG0Z7B+D3DE3st3LMdC9jWSv6BQzVU5T9bJgWECrqLP51VfWvMy/crzor3tbNY0wLx2f77u+ACTsELzKVOWcv3EEvO3jaZissFQ3Pk2L2K4zPv/LV8eObNs6v8q5NBEhfSqoboF3jD7rJQd4IpNT8CnsmaifgjSm1sLkFjsDs0vcYBamtQ4bQg1uGUA0RZ8Hj2fe63kwnhXaM069qn5LkbhQA9QfLycnRxBunUGq1OTSVZJYW2i4RDVvlI5pQjx+GF4HsQ4ElNc2WlWjRnEMHkEQr/z0+yjjCWoT0UuQgx5c+rRjqZ59vv8UKas9eyjkRspIt+yxkZ3hVi+7lbofi3srNPE72ODkuM4eokQQ5r81TQAU7sZ7eky8E2rOMvHCR6No44z0gfyBA4ObuTGXr0jIPkLVOC3ZHGqLHq4nJbSVvrWsjvvw0= ceph-97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:57.646 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4/oogTrwYQqZGuFhFGaoC/8pV3KBalx4+pwkGkE+r3kww0yOp49llyx2ROboMyRlwufZsWIy2cEG0Z7B+D3DE3st3LMdC9jWSv6BQzVU5T9bJgWECrqLP51VfWvMy/crzor3tbNY0wLx2f77u+ACTsELzKVOWcv3EEvO3jaZissFQ3Pk2L2K4zPv/LV8eObNs6v8q5NBEhfSqoboF3jD7rJQd4IpNT8CnsmaifgjSm1sLkFjsDs0vcYBamtQ4bQg1uGUA0RZ8Hj2fe63kwnhXaM069qn5LkbhQA9QfLycnRxBunUGq1OTSVZJYW2i4RDVvlI5pQjx+GF4HsQ4ElNc2WlWjRnEMHkEQr/z0+yjjCWoT0UuQgx5c+rRjqZ59vv8UKas9eyjkRspIt+yxkZ3hVi+7lbofi3srNPE72ODkuM4eokQQ5r81TQAU7sZ7eky8E2rOMvHCR6No44z0gfyBA4ObuTGXr0jIPkLVOC3ZHGqLHq4nJbSVvrWsjvvw0= ceph-97eecd66-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:03:57.685 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4/oogTrwYQqZGuFhFGaoC/8pV3KBalx4+pwkGkE+r3kww0yOp49llyx2ROboMyRlwufZsWIy2cEG0Z7B+D3DE3st3LMdC9jWSv6BQzVU5T9bJgWECrqLP51VfWvMy/crzor3tbNY0wLx2f77u+ACTsELzKVOWcv3EEvO3jaZissFQ3Pk2L2K4zPv/LV8eObNs6v8q5NBEhfSqoboF3jD7rJQd4IpNT8CnsmaifgjSm1sLkFjsDs0vcYBamtQ4bQg1uGUA0RZ8Hj2fe63kwnhXaM069qn5LkbhQA9QfLycnRxBunUGq1OTSVZJYW2i4RDVvlI5pQjx+GF4HsQ4ElNc2WlWjRnEMHkEQr/z0+yjjCWoT0UuQgx5c+rRjqZ59vv8UKas9eyjkRspIt+yxkZ3hVi+7lbofi3srNPE72ODkuM4eokQQ5r81TQAU7sZ7eky8E2rOMvHCR6No44z0gfyBA4ObuTGXr0jIPkLVOC3ZHGqLHq4nJbSVvrWsjvvw0= ceph-97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:57.697 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T23:03:57.937 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:03:58.060 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:03:57 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2026918705' entity='client.admin' 2024-04-24T23:04:00.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.152+0000 7fc47d49b700 1 -- 172.21.15.102:0/2156306253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 msgr2=0x7fc478100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:00.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.152+0000 7fc47d49b700 1 --2- 172.21.15.102:0/2156306253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 0x7fc478100ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc468009910 tx=0x7fc468009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:00.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.152+0000 7fc47d49b700 1 -- 172.21.15.102:0/2156306253 shutdown_connections 2024-04-24T23:04:00.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.152+0000 7fc47d49b700 1 --2- 172.21.15.102:0/2156306253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 0x7fc478100ff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.152+0000 7fc47d49b700 1 -- 172.21.15.102:0/2156306253 >> 172.21.15.102:0/2156306253 conn(0x7fc4780fc190 msgr2=0x7fc4780fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:00.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.152+0000 7fc47d49b700 1 -- 172.21.15.102:0/2156306253 shutdown_connections 2024-04-24T23:04:00.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.152+0000 7fc47d49b700 1 -- 172.21.15.102:0/2156306253 wait complete. 2024-04-24T23:04:00.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.153+0000 7fc47d49b700 1 Processor -- start 2024-04-24T23:04:00.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.153+0000 7fc47d49b700 1 -- start start 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.153+0000 7fc47d49b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 0x7fc47806aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.153+0000 7fc47d49b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc478106b10 con 0x7fc478100bd0 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.153+0000 7fc476ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 0x7fc47806aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.153+0000 7fc476ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 0x7fc47806aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53354/0 (socket says 172.21.15.102:53354) 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.153+0000 7fc476ffd700 1 -- 172.21.15.102:0/1535772945 learned_addr learned my addr 172.21.15.102:0/1535772945 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.153+0000 7fc476ffd700 1 -- 172.21.15.102:0/1535772945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47819a050 con 0x7fc478100bd0 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.154+0000 7fc476ffd700 1 --2- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 0x7fc47806aa40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc468005800 tx=0x7fc468009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.154+0000 7fc45ffff700 1 -- 172.21.15.102:0/1535772945 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc468027070 con 0x7fc478100bd0 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.154+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47819a220 con 0x7fc478100bd0 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.154+0000 7fc45ffff700 1 -- 172.21.15.102:0/1535772945 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc46800be10 con 0x7fc478100bd0 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.154+0000 7fc45ffff700 1 -- 172.21.15.102:0/1535772945 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc46800f460 con 0x7fc478100bd0 2024-04-24T23:04:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.154+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47819a5c0 con 0x7fc478100bd0 2024-04-24T23:04:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.155+0000 7fc45ffff700 1 -- 172.21.15.102:0/1535772945 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc46802a070 con 0x7fc478100bd0 2024-04-24T23:04:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.155+0000 7fc45ffff700 1 --2- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc4600390f0 0x7fc46003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.155+0000 7fc45ffff700 1 -- 172.21.15.102:0/1535772945 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc46801e030 con 0x7fc478100bd0 2024-04-24T23:04:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.155+0000 7fc4767fc700 1 --2- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc4600390f0 0x7fc46003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.155+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc478107290 con 0x7fc478100bd0 2024-04-24T23:04:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.155+0000 7fc4767fc700 1 --2- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc4600390f0 0x7fc46003b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc46c006fd0 tx=0x7fc46c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.160+0000 7fc45ffff700 1 -- 172.21.15.102:0/1535772945 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc468031070 con 0x7fc478100bd0 2024-04-24T23:04:00.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:00.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:00.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:00.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:00.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:00.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:00.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.361+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fc47802dca0 con 0x7fc478100bd0 2024-04-24T23:04:00.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.363+0000 7fc45ffff700 1 -- 172.21.15.102:0/1535772945 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fc468013070 con 0x7fc478100bd0 2024-04-24T23:04:00.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.365+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc4600390f0 msgr2=0x7fc46003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:00.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.365+0000 7fc47d49b700 1 --2- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc4600390f0 0x7fc46003b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc46c006fd0 tx=0x7fc46c006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:00.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.365+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 msgr2=0x7fc47806aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:00.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.365+0000 7fc47d49b700 1 --2- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 0x7fc47806aa40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc468005800 tx=0x7fc468009f60 comp rx=0 tx=0).stop 2024-04-24T23:04:00.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.365+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 shutdown_connections 2024-04-24T23:04:00.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.365+0000 7fc47d49b700 1 --2- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc4600390f0 0x7fc46003b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.365+0000 7fc47d49b700 1 --2- 172.21.15.102:0/1535772945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc478100bd0 0x7fc47806aa40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:00.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.365+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 >> 172.21.15.102:0/1535772945 conn(0x7fc4780fc190 msgr2=0x7fc4780fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:00.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.365+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 shutdown_connections 2024-04-24T23:04:00.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:00.366+0000 7fc47d49b700 1 -- 172.21.15.102:0/1535772945 wait complete. 2024-04-24T23:04:01.079 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T23:04:01.079 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T23:04:01.324 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:01.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:01 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1535772945' entity='client.admin' 2024-04-24T23:04:01.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:01 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:01.721 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:01 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:03.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.495+0000 7f7adb59e700 1 -- 172.21.15.102:0/1929871407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 msgr2=0x7f7adc104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.495+0000 7f7adb59e700 1 --2- 172.21.15.102:0/1929871407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 0x7f7adc104930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009910 tx=0x7f7acc009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:03.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.495+0000 7f7adb59e700 1 -- 172.21.15.102:0/1929871407 shutdown_connections 2024-04-24T23:04:03.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.495+0000 7f7adb59e700 1 --2- 172.21.15.102:0/1929871407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 0x7f7adc104930 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.495+0000 7f7adb59e700 1 -- 172.21.15.102:0/1929871407 >> 172.21.15.102:0/1929871407 conn(0x7f7adc0fbf50 msgr2=0x7f7adc0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:03.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.495+0000 7f7adb59e700 1 -- 172.21.15.102:0/1929871407 shutdown_connections 2024-04-24T23:04:03.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.495+0000 7f7adb59e700 1 -- 172.21.15.102:0/1929871407 wait complete. 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.496+0000 7f7adb59e700 1 Processor -- start 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.496+0000 7f7adb59e700 1 -- start start 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.496+0000 7f7adb59e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 0x7f7adc197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.496+0000 7f7adb59e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc107990 con 0x7f7adc102540 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.496+0000 7f7ada59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 0x7f7adc197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.496+0000 7f7ada59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 0x7f7adc197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53380/0 (socket says 172.21.15.102:53380) 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.496+0000 7f7ada59c700 1 -- 172.21.15.102:0/4277744163 learned_addr learned my addr 172.21.15.102:0/4277744163 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.497+0000 7f7ada59c700 1 -- 172.21.15.102:0/4277744163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc1975c0 con 0x7f7adc102540 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.497+0000 7f7ada59c700 1 --2- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 0x7f7adc197080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7acc005b40 tx=0x7f7acc004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.497+0000 7f7ad3fff700 1 -- 172.21.15.102:0/4277744163 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7acc027070 con 0x7f7adc102540 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.497+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc1978b0 con 0x7f7adc102540 2024-04-24T23:04:03.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.497+0000 7f7ad3fff700 1 -- 172.21.15.102:0/4277744163 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7acc00be10 con 0x7f7adc102540 2024-04-24T23:04:03.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.497+0000 7f7ad3fff700 1 -- 172.21.15.102:0/4277744163 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7acc022de0 con 0x7f7adc102540 2024-04-24T23:04:03.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.497+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc19a540 con 0x7f7adc102540 2024-04-24T23:04:03.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.498+0000 7f7ad3fff700 1 -- 172.21.15.102:0/4277744163 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7acc02a070 con 0x7f7adc102540 2024-04-24T23:04:03.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.498+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7adc108110 con 0x7f7adc102540 2024-04-24T23:04:03.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.498+0000 7f7ad3fff700 1 --2- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ac8039150 0x7f7ac803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:03.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.498+0000 7f7ad3fff700 1 -- 172.21.15.102:0/4277744163 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7acc01e030 con 0x7f7adc102540 2024-04-24T23:04:03.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.498+0000 7f7ad1d9b700 1 --2- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ac8039150 0x7f7ac803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:03.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.499+0000 7f7ad1d9b700 1 --2- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ac8039150 0x7f7ac803b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4006fd0 tx=0x7f7ac4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:03.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.503+0000 7f7ad3fff700 1 -- 172.21.15.102:0/4277744163 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7acc013070 con 0x7f7adc102540 2024-04-24T23:04:03.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.704+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f7adc02dd20 con 0x7f7ac8039150 2024-04-24T23:04:03.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.708+0000 7f7ad3fff700 1 -- 172.21.15.102:0/4277744163 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7adc02dd20 con 0x7f7ac8039150 2024-04-24T23:04:03.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ac8039150 msgr2=0x7f7ac803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 --2- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ac8039150 0x7f7ac803b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4006fd0 tx=0x7f7ac4006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:03.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 msgr2=0x7f7adc197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:03.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 --2- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 0x7f7adc197080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7acc005b40 tx=0x7f7acc004f80 comp rx=0 tx=0).stop 2024-04-24T23:04:03.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 shutdown_connections 2024-04-24T23:04:03.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 --2- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ac8039150 0x7f7ac803b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 --2- 172.21.15.102:0/4277744163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7adc102540 0x7f7adc197080 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:03.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 >> 172.21.15.102:0/4277744163 conn(0x7f7adc0fbf50 msgr2=0x7f7adc0fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:03.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 shutdown_connections 2024-04-24T23:04:03.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:03.710+0000 7f7adb59e700 1 -- 172.21.15.102:0/4277744163 wait complete. 2024-04-24T23:04:04.503 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-04-24T23:04:04.504 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T23:04:04.504 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:04:04.522 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T23:04:04.522 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:04.580 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-04-24T23:04:04.580 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch host add smithi161 2024-04-24T23:04:04.816 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:04.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:04.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:04.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:04.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:04.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:04.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-24T23:04:04.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:04.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:04.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:04.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:04.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:06.006 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:05 smithi102 ceph-mon[29216]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:04:06.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.335+0000 7fc409c40700 1 -- 172.21.15.102:0/2356929512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 msgr2=0x7fc404103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.335+0000 7fc409c40700 1 --2- 172.21.15.102:0/2356929512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 0x7fc404103290 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009910 tx=0x7fc3f4009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:06.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.335+0000 7fc409c40700 1 -- 172.21.15.102:0/2356929512 shutdown_connections 2024-04-24T23:04:06.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.335+0000 7fc409c40700 1 --2- 172.21.15.102:0/2356929512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 0x7fc404103290 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.335+0000 7fc409c40700 1 -- 172.21.15.102:0/2356929512 >> 172.21.15.102:0/2356929512 conn(0x7fc4040fc660 msgr2=0x7fc4040fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:06.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.335+0000 7fc409c40700 1 -- 172.21.15.102:0/2356929512 shutdown_connections 2024-04-24T23:04:06.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.335+0000 7fc409c40700 1 -- 172.21.15.102:0/2356929512 wait complete. 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.335+0000 7fc409c40700 1 Processor -- start 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc409c40700 1 -- start start 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc409c40700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 0x7fc404100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc409c40700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc40410c1b0 con 0x7fc404102eb0 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc4037fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 0x7fc404100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc4037fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 0x7fc404100b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53384/0 (socket says 172.21.15.102:53384) 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc4037fe700 1 -- 172.21.15.102:0/4234113202 learned_addr learned my addr 172.21.15.102:0/4234113202 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc4037fe700 1 -- 172.21.15.102:0/4234113202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc404102a00 con 0x7fc404102eb0 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc4037fe700 1 --2- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 0x7fc404100b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40056d0 tx=0x7fc3f40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc400ff9700 1 -- 172.21.15.102:0/4234113202 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f4026070 con 0x7fc404102eb0 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc400ff9700 1 -- 172.21.15.102:0/4234113202 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3f4004370 con 0x7fc404102eb0 2024-04-24T23:04:06.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.336+0000 7fc400ff9700 1 -- 172.21.15.102:0/4234113202 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f40039b0 con 0x7fc404102eb0 2024-04-24T23:04:06.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.337+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc404101050 con 0x7fc404102eb0 2024-04-24T23:04:06.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.337+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc404101510 con 0x7fc404102eb0 2024-04-24T23:04:06.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.337+0000 7fc400ff9700 1 -- 172.21.15.102:0/4234113202 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc3f4029070 con 0x7fc404102eb0 2024-04-24T23:04:06.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.337+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40406a870 con 0x7fc404102eb0 2024-04-24T23:04:06.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.338+0000 7fc400ff9700 1 --2- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3ec039150 0x7fc3ec03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.338+0000 7fc400ff9700 1 -- 172.21.15.102:0/4234113202 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc3f401e070 con 0x7fc404102eb0 2024-04-24T23:04:06.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.338+0000 7fc402ffd700 1 --2- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3ec039150 0x7fc3ec03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.338+0000 7fc402ffd700 1 --2- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3ec039150 0x7fc3ec03b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8006fd0 tx=0x7fc3f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:06.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.343+0000 7fc400ff9700 1 -- 172.21.15.102:0/4234113202 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3f4013070 con 0x7fc404102eb0 2024-04-24T23:04:06.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:06.545+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) -- 0x7fc4041020b0 con 0x7fc3ec039150 2024-04-24T23:04:07.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:07 smithi102 ceph-mon[29216]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:07.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:07 smithi102 ceph-mon[29216]: Deploying cephadm binary to smithi161 2024-04-24T23:04:08.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:08 smithi102 ceph-mon[29216]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:10.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.644+0000 7fc400ff9700 1 -- 172.21.15.102:0/4234113202 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc4041020b0 con 0x7fc3ec039150 2024-04-24T23:04:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-04-24T23:04:10.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3ec039150 msgr2=0x7fc3ec03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:10.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 --2- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3ec039150 0x7fc3ec03b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8006fd0 tx=0x7fc3f8006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:10.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 msgr2=0x7fc404100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:10.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 --2- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 0x7fc404100b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40056d0 tx=0x7fc3f40057b0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 shutdown_connections 2024-04-24T23:04:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 --2- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3ec039150 0x7fc3ec03b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 --2- 172.21.15.102:0/4234113202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc404102eb0 0x7fc404100b10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 >> 172.21.15.102:0/4234113202 conn(0x7fc4040fc660 msgr2=0x7fc404103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 shutdown_connections 2024-04-24T23:04:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:10.647+0000 7fc409c40700 1 -- 172.21.15.102:0/4234113202 wait complete. 2024-04-24T23:04:11.202 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:04:11.436 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:11.557 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:11 smithi102 ceph-mon[29216]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:11.558 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:11.558 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:11.558 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:12.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:12 smithi102 ceph-mon[29216]: Added host smithi161 2024-04-24T23:04:12.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:12 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.896+0000 7f3ee1484700 1 -- 172.21.15.102:0/2734780248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 msgr2=0x7f3edc106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.896+0000 7f3ee1484700 1 --2- 172.21.15.102:0/2734780248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 0x7f3edc106fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009910 tx=0x7f3ecc009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.896+0000 7f3ee1484700 1 -- 172.21.15.102:0/2734780248 shutdown_connections 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.896+0000 7f3ee1484700 1 --2- 172.21.15.102:0/2734780248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 0x7f3edc106fc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.896+0000 7f3ee1484700 1 -- 172.21.15.102:0/2734780248 >> 172.21.15.102:0/2734780248 conn(0x7f3edc077800 msgr2=0x7f3edc0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.896+0000 7f3ee1484700 1 -- 172.21.15.102:0/2734780248 shutdown_connections 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.896+0000 7f3ee1484700 1 -- 172.21.15.102:0/2734780248 wait complete. 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.897+0000 7f3ee1484700 1 Processor -- start 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.897+0000 7f3ee1484700 1 -- start start 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.897+0000 7f3ee1484700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 0x7f3edc196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:12.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.897+0000 7f3ee1484700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc10b050 con 0x7f3edc106be0 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.897+0000 7f3edaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 0x7f3edc196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.897+0000 7f3edaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 0x7f3edc196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34280/0 (socket says 172.21.15.102:34280) 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.897+0000 7f3edaffd700 1 -- 172.21.15.102:0/238358516 learned_addr learned my addr 172.21.15.102:0/238358516 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.898+0000 7f3edaffd700 1 -- 172.21.15.102:0/238358516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc196ad0 con 0x7f3edc106be0 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.898+0000 7f3edaffd700 1 --2- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 0x7f3edc196590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc006010 tx=0x7f3ecc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.898+0000 7f3ec3fff700 1 -- 172.21.15.102:0/238358516 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ecc026070 con 0x7f3edc106be0 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.898+0000 7f3ec3fff700 1 -- 172.21.15.102:0/238358516 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ecc00bc00 con 0x7f3edc106be0 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.898+0000 7f3ec3fff700 1 -- 172.21.15.102:0/238358516 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ecc003930 con 0x7f3edc106be0 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.898+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc196d60 con 0x7f3edc106be0 2024-04-24T23:04:12.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.898+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc1971c0 con 0x7f3edc106be0 2024-04-24T23:04:12.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.899+0000 7f3ec3fff700 1 -- 172.21.15.102:0/238358516 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3ecc029070 con 0x7f3edc106be0 2024-04-24T23:04:12.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.899+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc1021e0 con 0x7f3edc106be0 2024-04-24T23:04:12.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.899+0000 7f3ec3fff700 1 --2- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ec4039150 0x7f3ec403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:12.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.899+0000 7f3ec3fff700 1 -- 172.21.15.102:0/238358516 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3ecc01e070 con 0x7f3edc106be0 2024-04-24T23:04:12.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.899+0000 7f3eda7fc700 1 --2- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ec4039150 0x7f3ec403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:12.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.900+0000 7f3eda7fc700 1 --2- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ec4039150 0x7f3ec403b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006fd0 tx=0x7f3ed0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:12.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:12.904+0000 7f3ec3fff700 1 -- 172.21.15.102:0/238358516 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ecc013070 con 0x7f3edc106be0 2024-04-24T23:04:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.107+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3edc197960 con 0x7f3ec4039150 2024-04-24T23:04:13.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.108+0000 7f3ec3fff700 1 -- 172.21.15.102:0/238358516 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f3edc197960 con 0x7f3ec4039150 2024-04-24T23:04:13.110 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:04:13.110 INFO:teuthology.orchestra.run.smithi102.stdout:[{"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-04-24T23:04:13.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.110+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ec4039150 msgr2=0x7f3ec403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:13.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.110+0000 7f3ee1484700 1 --2- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ec4039150 0x7f3ec403b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006fd0 tx=0x7f3ed0006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.110+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 msgr2=0x7f3edc196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.110+0000 7f3ee1484700 1 --2- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 0x7f3edc196590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc006010 tx=0x7f3ecc005780 comp rx=0 tx=0).stop 2024-04-24T23:04:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.110+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 shutdown_connections 2024-04-24T23:04:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.110+0000 7f3ee1484700 1 --2- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ec4039150 0x7f3ec403b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.110+0000 7f3ee1484700 1 --2- 172.21.15.102:0/238358516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3edc106be0 0x7f3edc196590 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.110+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 >> 172.21.15.102:0/238358516 conn(0x7f3edc077800 msgr2=0x7f3edc0fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.111+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 shutdown_connections 2024-04-24T23:04:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:13.111+0000 7f3ee1484700 1 -- 172.21.15.102:0/238358516 wait complete. 2024-04-24T23:04:13.457 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:13 smithi102 ceph-mon[29216]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:13.708 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-04-24T23:04:13.709 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T23:04:13.709 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:04:13.726 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T23:04:13.727 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:13.785 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-04-24T23:04:13.785 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch host add smithi183 2024-04-24T23:04:14.025 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:14.478 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:14 smithi102 ceph-mon[29216]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:04:14.478 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:14 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:15.376 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:15 smithi102 ceph-mon[29216]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:15.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.594+0000 7f1b3fc94700 1 -- 172.21.15.102:0/2740059787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 msgr2=0x7f1b38102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:15.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.594+0000 7f1b3fc94700 1 --2- 172.21.15.102:0/2740059787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 0x7f1b38102290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009910 tx=0x7f1b34009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:15.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.594+0000 7f1b3fc94700 1 -- 172.21.15.102:0/2740059787 shutdown_connections 2024-04-24T23:04:15.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.594+0000 7f1b3fc94700 1 --2- 172.21.15.102:0/2740059787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 0x7f1b38102290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:15.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.594+0000 7f1b3fc94700 1 -- 172.21.15.102:0/2740059787 >> 172.21.15.102:0/2740059787 conn(0x7f1b380fd760 msgr2=0x7f1b380ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:15.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.594+0000 7f1b3fc94700 1 -- 172.21.15.102:0/2740059787 shutdown_connections 2024-04-24T23:04:15.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.595+0000 7f1b3fc94700 1 -- 172.21.15.102:0/2740059787 wait complete. 2024-04-24T23:04:15.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.595+0000 7f1b3fc94700 1 Processor -- start 2024-04-24T23:04:15.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.595+0000 7f1b3fc94700 1 -- start start 2024-04-24T23:04:15.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.595+0000 7f1b3fc94700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 0x7f1b38197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:15.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.595+0000 7f1b3fc94700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3810c130 con 0x7f1b38101eb0 2024-04-24T23:04:15.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.595+0000 7f1b3da30700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 0x7f1b38197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:15.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.595+0000 7f1b3da30700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 0x7f1b38197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34296/0 (socket says 172.21.15.102:34296) 2024-04-24T23:04:15.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.595+0000 7f1b3da30700 1 -- 172.21.15.102:0/3147921366 learned_addr learned my addr 172.21.15.102:0/3147921366 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:04:15.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.596+0000 7f1b3da30700 1 -- 172.21.15.102:0/3147921366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b38197ba0 con 0x7f1b38101eb0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.596+0000 7f1b3da30700 1 --2- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 0x7f1b38197660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b340056d0 tx=0x7f1b340057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.596+0000 7f1b2effd700 1 -- 172.21.15.102:0/3147921366 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b34026070 con 0x7f1b38101eb0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.596+0000 7f1b2effd700 1 -- 172.21.15.102:0/3147921366 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b3400bba0 con 0x7f1b38101eb0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.596+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b38197e90 con 0x7f1b38101eb0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.596+0000 7f1b2effd700 1 -- 172.21.15.102:0/3147921366 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b340037f0 con 0x7f1b38101eb0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.596+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3819bf30 con 0x7f1b38101eb0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.597+0000 7f1b2effd700 1 -- 172.21.15.102:0/3147921366 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1b34029070 con 0x7f1b38101eb0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.597+0000 7f1b2effd700 1 --2- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1b24039150 0x7f1b2403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.597+0000 7f1b2effd700 1 -- 172.21.15.102:0/3147921366 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1b3401e070 con 0x7f1b38101eb0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.597+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b38103390 con 0x7f1b38101eb0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.597+0000 7f1b3d22f700 1 --2- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1b24039150 0x7f1b2403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:15.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.597+0000 7f1b3d22f700 1 --2- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1b24039150 0x7f1b2403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1b28006fd0 tx=0x7f1b28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:15.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.602+0000 7f1b2effd700 1 -- 172.21.15.102:0/3147921366 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1b3407b740 con 0x7f1b38101eb0 2024-04-24T23:04:15.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:15.801+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7f1b381989c0 con 0x7f1b24039150 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:17 smithi102 ceph-mon[29216]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-24T23:04:18.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:18 smithi102 ceph-mon[29216]: Deploying cephadm binary to smithi183 2024-04-24T23:04:18.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:18 smithi102 ceph-mon[29216]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:18.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:18 smithi102 ceph-mon[29216]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:19 smithi102 ceph-mon[29216]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:19.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.898+0000 7f1b2effd700 1 -- 172.21.15.102:0/3147921366 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f1b381989c0 con 0x7f1b24039150 2024-04-24T23:04:19.900 INFO:teuthology.orchestra.run.smithi102.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-04-24T23:04:19.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.900+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1b24039150 msgr2=0x7f1b2403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.900+0000 7f1b3fc94700 1 --2- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1b24039150 0x7f1b2403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1b28006fd0 tx=0x7f1b28006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:19.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.900+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 msgr2=0x7f1b38197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.900+0000 7f1b3fc94700 1 --2- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 0x7f1b38197660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b340056d0 tx=0x7f1b340057b0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.900+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 shutdown_connections 2024-04-24T23:04:19.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.900+0000 7f1b3fc94700 1 --2- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1b24039150 0x7f1b2403b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.900+0000 7f1b3fc94700 1 --2- 172.21.15.102:0/3147921366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38101eb0 0x7f1b38197660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.900+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 >> 172.21.15.102:0/3147921366 conn(0x7f1b380fd760 msgr2=0x7f1b38103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:19.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.901+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 shutdown_connections 2024-04-24T23:04:19.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:19.901+0000 7f1b3fc94700 1 -- 172.21.15.102:0/3147921366 wait complete. 2024-04-24T23:04:20.504 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:20 smithi102 ceph-mon[29216]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:04:20.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:20.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:20.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:20.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:20.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:20.531 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:04:20.760 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:21 smithi102 ceph-mon[29216]: Added host smithi183 2024-04-24T23:04:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:21 smithi102 ceph-mon[29216]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:21 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:21 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.367+0000 7fb9670a6700 1 -- 172.21.15.102:0/3589499996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 msgr2=0x7fb960102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.367+0000 7fb9670a6700 1 --2- 172.21.15.102:0/3589499996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 0x7fb960102250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb954009910 tx=0x7fb954009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.367+0000 7fb9670a6700 1 -- 172.21.15.102:0/3589499996 shutdown_connections 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.367+0000 7fb9670a6700 1 --2- 172.21.15.102:0/3589499996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 0x7fb960102250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.367+0000 7fb9670a6700 1 -- 172.21.15.102:0/3589499996 >> 172.21.15.102:0/3589499996 conn(0x7fb9600fd740 msgr2=0x7fb9600ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.367+0000 7fb9670a6700 1 -- 172.21.15.102:0/3589499996 shutdown_connections 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.367+0000 7fb9670a6700 1 -- 172.21.15.102:0/3589499996 wait complete. 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.368+0000 7fb9670a6700 1 Processor -- start 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.368+0000 7fb9670a6700 1 -- start start 2024-04-24T23:04:22.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.368+0000 7fb9670a6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 0x7fb960197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.368+0000 7fb9670a6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb96010c0f0 con 0x7fb960101e70 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.368+0000 7fb964e42700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 0x7fb960197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.368+0000 7fb964e42700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 0x7fb960197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44500/0 (socket says 172.21.15.102:44500) 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.368+0000 7fb964e42700 1 -- 172.21.15.102:0/1471008987 learned_addr learned my addr 172.21.15.102:0/1471008987 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.369+0000 7fb964e42700 1 -- 172.21.15.102:0/1471008987 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb960197b90 con 0x7fb960101e70 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.369+0000 7fb964e42700 1 --2- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 0x7fb960197650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb954006010 tx=0x7fb954005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.369+0000 7fb95dffb700 1 -- 172.21.15.102:0/1471008987 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb954026070 con 0x7fb960101e70 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.369+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb960197e80 con 0x7fb960101e70 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.369+0000 7fb95dffb700 1 -- 172.21.15.102:0/1471008987 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb95400bba0 con 0x7fb960101e70 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.369+0000 7fb95dffb700 1 -- 172.21.15.102:0/1471008987 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9540038a0 con 0x7fb960101e70 2024-04-24T23:04:22.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.369+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96019bf20 con 0x7fb960101e70 2024-04-24T23:04:22.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.370+0000 7fb95dffb700 1 -- 172.21.15.102:0/1471008987 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fb954029070 con 0x7fb960101e70 2024-04-24T23:04:22.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.370+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb960103350 con 0x7fb960101e70 2024-04-24T23:04:22.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.370+0000 7fb95dffb700 1 --2- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb9500390f0 0x7fb95003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:22.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.370+0000 7fb95dffb700 1 -- 172.21.15.102:0/1471008987 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb95401e070 con 0x7fb960101e70 2024-04-24T23:04:22.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.370+0000 7fb95ffff700 1 --2- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb9500390f0 0x7fb95003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:22.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.371+0000 7fb95ffff700 1 --2- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb9500390f0 0x7fb95003b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb94c006fd0 tx=0x7fb94c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:22.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.375+0000 7fb95dffb700 1 -- 172.21.15.102:0/1471008987 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb954013070 con 0x7fb960101e70 2024-04-24T23:04:22.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.576+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9601989b0 con 0x7fb9500390f0 2024-04-24T23:04:22.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.578+0000 7fb95dffb700 1 -- 172.21.15.102:0/1471008987 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fb9601989b0 con 0x7fb9500390f0 2024-04-24T23:04:22.578 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:04:22.579 INFO:teuthology.orchestra.run.smithi102.stdout:[{"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb9500390f0 msgr2=0x7fb95003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 --2- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb9500390f0 0x7fb95003b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb94c006fd0 tx=0x7fb94c006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 msgr2=0x7fb960197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 --2- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 0x7fb960197650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb954006010 tx=0x7fb954005780 comp rx=0 tx=0).stop 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 shutdown_connections 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 --2- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb9500390f0 0x7fb95003b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 --2- 172.21.15.102:0/1471008987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb960101e70 0x7fb960197650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 >> 172.21.15.102:0/1471008987 conn(0x7fb9600fd740 msgr2=0x7fb960103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 shutdown_connections 2024-04-24T23:04:22.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:22.580+0000 7fb9670a6700 1 -- 172.21.15.102:0/1471008987 wait complete. 2024-04-24T23:04:23.197 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T23:04:23.198 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T23:04:23.430 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:23.559 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:23 smithi102 ceph-mon[29216]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:23.559 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:24.562 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:24 smithi102 ceph-mon[29216]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:04:25.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.011+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/1661429710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 msgr2=0x7fbc38106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.011+0000 7fbc3f4ee700 1 --2- 172.21.15.102:0/1661429710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 0x7fbc38106fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009910 tx=0x7fbc28009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.011+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/1661429710 shutdown_connections 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.011+0000 7fbc3f4ee700 1 --2- 172.21.15.102:0/1661429710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 0x7fbc38106fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.011+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/1661429710 >> 172.21.15.102:0/1661429710 conn(0x7fbc380fc680 msgr2=0x7fbc380feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.011+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/1661429710 shutdown_connections 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.011+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/1661429710 wait complete. 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.012+0000 7fbc3f4ee700 1 Processor -- start 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.012+0000 7fbc3f4ee700 1 -- start start 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.012+0000 7fbc3f4ee700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 0x7fbc381984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.012+0000 7fbc3f4ee700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3810b070 con 0x7fbc38106c00 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.012+0000 7fbc3d28a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 0x7fbc381984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:25.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.012+0000 7fbc3d28a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 0x7fbc381984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44512/0 (socket says 172.21.15.102:44512) 2024-04-24T23:04:25.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.012+0000 7fbc3d28a700 1 -- 172.21.15.102:0/3807268178 learned_addr learned my addr 172.21.15.102:0/3807268178 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:04:25.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.012+0000 7fbc3d28a700 1 -- 172.21.15.102:0/3807268178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc381989e0 con 0x7fbc38106c00 2024-04-24T23:04:25.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.012+0000 7fbc3d28a700 1 --2- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 0x7fbc381984a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc280056d0 tx=0x7fbc280057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:25.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.013+0000 7fbc2e7fc700 1 -- 172.21.15.102:0/3807268178 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc28026070 con 0x7fbc38106c00 2024-04-24T23:04:25.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.013+0000 7fbc2e7fc700 1 -- 172.21.15.102:0/3807268178 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc2800bc00 con 0x7fbc38106c00 2024-04-24T23:04:25.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.013+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc38198c70 con 0x7fbc38106c00 2024-04-24T23:04:25.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.013+0000 7fbc2e7fc700 1 -- 172.21.15.102:0/3807268178 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc280039b0 con 0x7fbc38106c00 2024-04-24T23:04:25.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.013+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc381990d0 con 0x7fbc38106c00 2024-04-24T23:04:25.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.013+0000 7fbc2e7fc700 1 -- 172.21.15.102:0/3807268178 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fbc28029070 con 0x7fbc38106c00 2024-04-24T23:04:25.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.014+0000 7fbc2e7fc700 1 --2- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbc24039150 0x7fbc2403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:25.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.014+0000 7fbc2e7fc700 1 -- 172.21.15.102:0/3807268178 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc2801e070 con 0x7fbc38106c00 2024-04-24T23:04:25.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.014+0000 7fbc3ca89700 1 --2- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbc24039150 0x7fbc2403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:25.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.014+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc38102200 con 0x7fbc38106c00 2024-04-24T23:04:25.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.014+0000 7fbc3ca89700 1 --2- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbc24039150 0x7fbc2403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbc34006fd0 tx=0x7fbc34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:25.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.019+0000 7fbc2e7fc700 1 -- 172.21.15.102:0/3807268178 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbc2807b740 con 0x7fbc38106c00 2024-04-24T23:04:25.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.220+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fbc38063890 con 0x7fbc38106c00 2024-04-24T23:04:25.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.279+0000 7fbc2e7fc700 1 -- 172.21.15.102:0/3807268178 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fbc28030070 con 0x7fbc38106c00 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbc24039150 msgr2=0x7fbc2403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 --2- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbc24039150 0x7fbc2403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbc34006fd0 tx=0x7fbc34006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 msgr2=0x7fbc381984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 --2- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 0x7fbc381984a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc280056d0 tx=0x7fbc280057b0 comp rx=0 tx=0).stop 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 shutdown_connections 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 --2- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbc24039150 0x7fbc2403b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 --2- 172.21.15.102:0/3807268178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc38106c00 0x7fbc381984a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 >> 172.21.15.102:0/3807268178 conn(0x7fbc380fc680 msgr2=0x7fbc380fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 shutdown_connections 2024-04-24T23:04:25.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:25.282+0000 7fbc3f4ee700 1 -- 172.21.15.102:0/3807268178 wait complete. 2024-04-24T23:04:25.284 INFO:teuthology.orchestra.run.smithi102.stderr:adjusted tunables profile to default 2024-04-24T23:04:25.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:25 smithi102 ceph-mon[29216]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:25.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:25 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3807268178' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:04:25.916 INFO:tasks.cephadm:Adding mon.a on smithi102 2024-04-24T23:04:25.916 INFO:tasks.cephadm:Adding mon.b on smithi161 2024-04-24T23:04:25.916 INFO:tasks.cephadm:Adding mon.c on smithi183 2024-04-24T23:04:25.916 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi102:172.21.15.102=a;smithi161:172.21.15.161=b;smithi183:172.21.15.183=c' 2024-04-24T23:04:26.150 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:26.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:26 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3807268178' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:04:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:26 smithi102 ceph-mon[29216]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:26 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:26 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:26 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:26 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:26 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:26 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:26 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:27 smithi102 ceph-mon[29216]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-24T23:04:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:27 smithi102 ceph-mon[29216]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:27 smithi102 ceph-mon[29216]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:27 smithi102 ceph-mon[29216]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:27.628 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.626+0000 7f8aea26b700 1 -- 172.21.15.183:0/523552353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 msgr2=0x7f8ae4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:27.628 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.626+0000 7f8aea26b700 1 --2- 172.21.15.183:0/523552353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 0x7f8ae4101180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009910 tx=0x7f8ad4009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:27.628 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.626+0000 7f8aea26b700 1 -- 172.21.15.183:0/523552353 shutdown_connections 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.626+0000 7f8aea26b700 1 --2- 172.21.15.183:0/523552353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 0x7f8ae4101180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.626+0000 7f8aea26b700 1 -- 172.21.15.183:0/523552353 >> 172.21.15.183:0/523552353 conn(0x7f8ae40fc670 msgr2=0x7f8ae40fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.626+0000 7f8aea26b700 1 -- 172.21.15.183:0/523552353 shutdown_connections 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.626+0000 7f8aea26b700 1 -- 172.21.15.183:0/523552353 wait complete. 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.626+0000 7f8aea26b700 1 Processor -- start 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.626+0000 7f8aea26b700 1 -- start start 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.627+0000 7f8aea26b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 0x7f8ae4102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.627+0000 7f8aea26b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae410b020 con 0x7f8ae4100da0 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.627+0000 7f8ae3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 0x7f8ae4102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.627+0000 7f8ae3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 0x7f8ae4102840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.183:53726/0 (socket says 172.21.15.183:53726) 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.627+0000 7f8ae3fff700 1 -- 172.21.15.183:0/1853970163 learned_addr learned my addr 172.21.15.183:0/1853970163 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:04:27.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.627+0000 7f8ae3fff700 1 -- 172.21.15.183:0/1853970163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae41046e0 con 0x7f8ae4100da0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.627+0000 7f8ae3fff700 1 --2- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 0x7f8ae4102840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40056d0 tx=0x7f8ad40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.627+0000 7f8ae17fa700 1 -- 172.21.15.183:0/1853970163 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ad4026070 con 0x7f8ae4100da0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.628+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae4102d80 con 0x7f8ae4100da0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.628+0000 7f8ae17fa700 1 -- 172.21.15.183:0/1853970163 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ad400bc00 con 0x7f8ae4100da0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.628+0000 7f8ae17fa700 1 -- 172.21.15.183:0/1853970163 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ad40039b0 con 0x7f8ae4100da0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.628+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae4103240 con 0x7f8ae4100da0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.628+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae4102280 con 0x7f8ae4100da0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.628+0000 7f8ae17fa700 1 -- 172.21.15.183:0/1853970163 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8ad4029070 con 0x7f8ae4100da0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.629+0000 7f8ae17fa700 1 --2- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8acc0390f0 0x7f8acc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.629+0000 7f8ae17fa700 1 -- 172.21.15.183:0/1853970163 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8ad401e070 con 0x7f8ae4100da0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.629+0000 7f8ae37fe700 1 --2- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8acc0390f0 0x7f8acc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:27.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.629+0000 7f8ae37fe700 1 --2- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8acc0390f0 0x7f8acc03b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006fd0 tx=0x7f8ad8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:27.635 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.634+0000 7f8ae17fa700 1 -- 172.21.15.183:0/1853970163 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8ad4013070 con 0x7f8ae4100da0 2024-04-24T23:04:27.840 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.838+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi102:172.21.15.102=a;smithi161:172.21.15.161=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}) -- 0x7f8ae4103d70 con 0x7f8acc0390f0 2024-04-24T23:04:27.846 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.845+0000 7f8ae17fa700 1 -- 172.21.15.183:0/1853970163 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8ae4103d70 con 0x7f8acc0390f0 2024-04-24T23:04:27.846 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8acc0390f0 msgr2=0x7f8acc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 --2- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8acc0390f0 0x7f8acc03b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006fd0 tx=0x7f8ad8006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 msgr2=0x7f8ae4102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 --2- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 0x7f8ae4102840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40056d0 tx=0x7f8ad40057b0 comp rx=0 tx=0).stop 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 shutdown_connections 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 --2- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8acc0390f0 0x7f8acc03b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 --2- 172.21.15.183:0/1853970163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ae4100da0 0x7f8ae4102840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 >> 172.21.15.183:0/1853970163 conn(0x7f8ae40fc670 msgr2=0x7f8ae4108ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 shutdown_connections 2024-04-24T23:04:27.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:27.847+0000 7f8aea26b700 1 -- 172.21.15.183:0/1853970163 wait complete. 2024-04-24T23:04:28.392 DEBUG:teuthology.orchestra.run.smithi161:mon.b> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T23:04:28.395 DEBUG:teuthology.orchestra.run.smithi183:mon.c> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T23:04:28.397 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:04:28.398 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:04:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:28 smithi102 ceph-mon[29216]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:04:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:29.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:29 smithi102 ceph-mon[29216]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi102:172.21.15.102=a;smithi161:172.21.15.161=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:29 smithi102 ceph-mon[29216]: Saving service mon spec with placement smithi102:172.21.15.102=a;smithi161:172.21.15.161=b;smithi183:172.21.15.183=c;count:3 2024-04-24T23:04:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:29 smithi102 ceph-mon[29216]: Deploying daemon mon.c on smithi183 2024-04-24T23:04:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:29 smithi102 ceph-mon[29216]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:30.695 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:04:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:31 smithi102 ceph-mon[29216]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:32.517 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.514+0000 7fae66082700 1 -- 172.21.15.183:0/3080241758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 msgr2=0x7fae60102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:32.517 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.514+0000 7fae66082700 1 --2- 172.21.15.183:0/3080241758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 0x7fae60102390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae50009910 tx=0x7fae50009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:32.517 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.514+0000 7fae66082700 1 -- 172.21.15.183:0/3080241758 shutdown_connections 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.514+0000 7fae66082700 1 --2- 172.21.15.183:0/3080241758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 0x7fae60102390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.514+0000 7fae66082700 1 -- 172.21.15.183:0/3080241758 >> 172.21.15.183:0/3080241758 conn(0x7fae600fd840 msgr2=0x7fae600ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.514+0000 7fae66082700 1 -- 172.21.15.183:0/3080241758 shutdown_connections 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.514+0000 7fae66082700 1 -- 172.21.15.183:0/3080241758 wait complete. 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.514+0000 7fae66082700 1 Processor -- start 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.515+0000 7fae66082700 1 -- start start 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.515+0000 7fae66082700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 0x7fae6019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.515+0000 7fae66082700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae6010c230 con 0x7fae60101fb0 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.515+0000 7fae5f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 0x7fae6019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.515+0000 7fae5f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 0x7fae6019d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.183:57936/0 (socket says 172.21.15.183:57936) 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.515+0000 7fae5f7fe700 1 -- 172.21.15.183:0/3035921376 learned_addr learned my addr 172.21.15.183:0/3035921376 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.515+0000 7fae5f7fe700 1 -- 172.21.15.183:0/3035921376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6019d940 con 0x7fae60101fb0 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.515+0000 7fae5f7fe700 1 --2- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 0x7fae6019d400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fae500056d0 tx=0x7fae500057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:32.518 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.516+0000 7fae5cff9700 1 -- 172.21.15.183:0/3035921376 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae50026070 con 0x7fae60101fb0 2024-04-24T23:04:32.519 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.516+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6019dc30 con 0x7fae60101fb0 2024-04-24T23:04:32.519 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.516+0000 7fae5cff9700 1 -- 172.21.15.183:0/3035921376 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae5000bba0 con 0x7fae60101fb0 2024-04-24T23:04:32.519 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.516+0000 7fae5cff9700 1 -- 172.21.15.183:0/3035921376 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae500037f0 con 0x7fae60101fb0 2024-04-24T23:04:32.519 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.516+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae601975c0 con 0x7fae60101fb0 2024-04-24T23:04:32.519 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.517+0000 7fae5cff9700 1 -- 172.21.15.183:0/3035921376 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fae50029070 con 0x7fae60101fb0 2024-04-24T23:04:32.520 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.516+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae60103490 con 0x7fae60101fb0 2024-04-24T23:04:32.520 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.517+0000 7fae5cff9700 1 --2- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fae48039150 0x7fae4803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:32.520 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.517+0000 7fae5cff9700 1 -- 172.21.15.183:0/3035921376 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fae5001e070 con 0x7fae60101fb0 2024-04-24T23:04:32.520 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.517+0000 7fae5effd700 1 --2- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fae48039150 0x7fae4803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:32.520 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.517+0000 7fae5effd700 1 --2- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fae48039150 0x7fae4803b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fae54006fd0 tx=0x7fae54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:32.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.522+0000 7fae5cff9700 1 -- 172.21.15.183:0/3035921376 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fae50013070 con 0x7fae60101fb0 2024-04-24T23:04:32.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.814+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fae60063890 con 0x7fae60101fb0 2024-04-24T23:04:32.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.815+0000 7fae5cff9700 1 -- 172.21.15.183:0/3035921376 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fae50030020 con 0x7fae60101fb0 2024-04-24T23:04:32.816 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:04:32.816 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:02:41.198045Z","created":"2024-04-24T23:02:41.198045Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:04:32.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fae48039150 msgr2=0x7fae4803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:32.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 --2- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fae48039150 0x7fae4803b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fae54006fd0 tx=0x7fae54006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:32.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 msgr2=0x7fae6019d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:32.819 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 --2- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 0x7fae6019d400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fae500056d0 tx=0x7fae500057b0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.819 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 shutdown_connections 2024-04-24T23:04:32.819 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 --2- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fae48039150 0x7fae4803b610 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.819 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 --2- 172.21.15.183:0/3035921376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae60101fb0 0x7fae6019d400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.819 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 >> 172.21.15.183:0/3035921376 conn(0x7fae600fd840 msgr2=0x7fae60103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:32.819 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 shutdown_connections 2024-04-24T23:04:32.819 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:32.817+0000 7fae66082700 1 -- 172.21.15.183:0/3035921376 wait complete. 2024-04-24T23:04:32.819 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2024-04-24T23:04:32.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:32 smithi102 ceph-mon[29216]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:32.939 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T23:04:32.939 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T23:04:32.939 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T23:04:32.939 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T23:04:32.939 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T23:04:33.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:04:33.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:04:33.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:04:33.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:04:33.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:04:33.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Deploying cephadm binary to smithi183 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:33.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Added host smithi183 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3807268178' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3807268178' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:33.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:33.266 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:33.267 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi102:172.21.15.102=a;smithi161:172.21.15.161=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:33.267 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Saving service mon spec with placement smithi102:172.21.15.102=a;smithi161:172.21.15.161=b;smithi183:172.21.15.183=c;count:3 2024-04-24T23:04:33.267 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: Deploying daemon mon.c on smithi183 2024-04-24T23:04:33.267 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.267 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.267 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:33.267 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 ceph-mon[33137]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T23:04:33.267 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 bash[33104]: daac70bc659d7763018fc46f08325a8e3659d69a77a8d21bb1922634206a966e 2024-04-24T23:04:33.267 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:32 smithi183 systemd[1]: Started Ceph mon.c for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:04:34.610 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:04:34.610 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:04:34.849 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:04:38.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:38.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: mon.a calling monitor election 2024-04-24T23:04:38.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:38.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: mon.c calling monitor election 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: monmap epoch 2 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: fsid 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: last_changed 2024-04-24T23:04:32.943793+0000 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: created 2024-04-24T23:02:41.198045+0000 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: min_mon_release 19 (squid) 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: election_strategy: 1 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-04-24T23:04:38.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-24T23:04:38.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: fsmap 2024-04-24T23:04:38.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:38.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: mgrmap e13: x(active, since 49s) 2024-04-24T23:04:38.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: overall HEALTH_OK 2024-04-24T23:04:38.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:38.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: mon.a calling monitor election 2024-04-24T23:04:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: mon.c calling monitor election 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: monmap epoch 2 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: fsid 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: last_changed 2024-04-24T23:04:32.943793+0000 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: created 2024-04-24T23:02:41.198045+0000 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: min_mon_release 19 (squid) 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: election_strategy: 1 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-04-24T23:04:38.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-24T23:04:38.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: fsmap 2024-04-24T23:04:38.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:38.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: mgrmap e13: x(active, since 49s) 2024-04-24T23:04:38.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: overall HEALTH_OK 2024-04-24T23:04:38.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:38.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:39 smithi102 ceph-mon[29216]: Deploying daemon mon.b on smithi161 2024-04-24T23:04:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:39 smithi102 ceph-mon[29216]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:39.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.382+0000 7f699ae3b700 1 -- 172.21.15.183:0/2307084737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840035c0 msgr2=0x7f6984005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.382+0000 7f699ae3b700 1 --2- 172.21.15.183:0/2307084737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840035c0 0x7f6984005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6994079470 tx=0x7f6988010ec0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.382+0000 7f699ae3b700 1 -- 172.21.15.183:0/2307084737 shutdown_connections 2024-04-24T23:04:39.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.382+0000 7f699ae3b700 1 --2- 172.21.15.183:0/2307084737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69840035c0 0x7f6984005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.382+0000 7f699ae3b700 1 -- 172.21.15.183:0/2307084737 >> 172.21.15.183:0/2307084737 conn(0x7f69940772e0 msgr2=0x7f69940776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.382+0000 7f699ae3b700 1 -- 172.21.15.183:0/2307084737 shutdown_connections 2024-04-24T23:04:39.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:39.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:39.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:39.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:39 smithi183 ceph-mon[33137]: Deploying daemon mon.b on smithi161 2024-04-24T23:04:39.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:39 smithi183 ceph-mon[33137]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:39.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.383+0000 7f699ae3b700 1 -- 172.21.15.183:0/2307084737 wait complete. 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.384+0000 7f699ae3b700 1 Processor -- start 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.384+0000 7f699ae3b700 1 -- start start 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f699ae3b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 0x7f699419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f699ae3b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699419d950 0x7f69941975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f699ae3b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6994107a90 con 0x7f699419d950 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f699ae3b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6994107910 con 0x7f69840035c0 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6998bd7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 0x7f699419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6998bd7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 0x7f699419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:34256/0 (socket says 172.21.15.183:34256) 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6998bd7700 1 -- 172.21.15.183:0/3742830441 learned_addr learned my addr 172.21.15.183:0/3742830441 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6998bd7700 1 -- 172.21.15.183:0/3742830441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 msgr2=0x7f699419d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6998bd7700 1 -- 172.21.15.183:0/3742830441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 msgr2=0x7f699419d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:04:39.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6998bd7700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 0x7f699419d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:04:39.388 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6993fff700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699419d950 0x7f69941975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.388 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6998bd7700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 0x7f699419d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:39.388 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6993fff700 1 -- 172.21.15.183:0/3742830441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 msgr2=0x7f699419d410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:39.388 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6993fff700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 0x7f699419d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.388 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.385+0000 7f6993fff700 1 -- 172.21.15.183:0/3742830441 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699419df90 con 0x7f699419d950 2024-04-24T23:04:39.388 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.386+0000 7f6993fff700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699419d950 0x7f69941975d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f698000b4d0 tx=0x7f698000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.389 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.386+0000 7f6991ffb700 1 -- 172.21.15.183:0/3742830441 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6980011630 con 0x7f699419d950 2024-04-24T23:04:39.389 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.386+0000 7f6991ffb700 1 -- 172.21.15.183:0/3742830441 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6980011c80 con 0x7f699419d950 2024-04-24T23:04:39.389 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.386+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6994197b10 con 0x7f699419d950 2024-04-24T23:04:39.389 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.386+0000 7f6991ffb700 1 -- 172.21.15.183:0/3742830441 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6980010da0 con 0x7f699419d950 2024-04-24T23:04:39.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.386+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69941980a0 con 0x7f699419d950 2024-04-24T23:04:39.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.387+0000 7f6991ffb700 1 -- 172.21.15.183:0/3742830441 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f6980018020 con 0x7f699419d950 2024-04-24T23:04:39.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.387+0000 7f6991ffb700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6978038ee0 0x7f697803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.387+0000 7f6991ffb700 1 -- 172.21.15.183:0/3742830441 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f698004ca30 con 0x7f699419d950 2024-04-24T23:04:39.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.387+0000 7f6998bd7700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6978038ee0 0x7f697803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.387+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6994101b10 con 0x7f699419d950 2024-04-24T23:04:39.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.388+0000 7f6998bd7700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6978038ee0 0x7f697803b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6988010a70 tx=0x7f6988005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.394 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.393+0000 7f6991ffb700 1 -- 172.21.15.183:0/3742830441 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f698001f020 con 0x7f699419d950 2024-04-24T23:04:39.679 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.678+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6994198b00 con 0x7f699419d950 2024-04-24T23:04:39.680 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.678+0000 7f6991ffb700 1 -- 172.21.15.183:0/3742830441 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f6980018570 con 0x7f699419d950 2024-04-24T23:04:39.680 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:04:39.680 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:04:32.943793Z","created":"2024-04-24T23:02:41.198045Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T23:04:39.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6978038ee0 msgr2=0x7f697803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6978038ee0 0x7f697803b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6988010a70 tx=0x7f6988005bc0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699419d950 msgr2=0x7f69941975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699419d950 0x7f69941975d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f698000b4d0 tx=0x7f698000b890 comp rx=0 tx=0).stop 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 shutdown_connections 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69840035c0 0x7f699419d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6978038ee0 0x7f697803b3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 --2- 172.21.15.183:0/3742830441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699419d950 0x7f69941975d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 >> 172.21.15.183:0/3742830441 conn(0x7f69940772e0 msgr2=0x7f69940ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 shutdown_connections 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:39.682+0000 7f699ae3b700 1 -- 172.21.15.183:0/3742830441 wait complete. 2024-04-24T23:04:39.684 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-04-24T23:04:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:40 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/3742830441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:04:40.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:40 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/3742830441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:04:41.235 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:04:41.235 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:04:41.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:41 smithi102 ceph-mon[29216]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:41.478 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:04:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:41 smithi183 ceph-mon[33137]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:42.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:41 smithi161 systemd[1]: Started Ceph mon.b for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:04:47.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:47.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: mon.a calling monitor election 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: mon.c calling monitor election 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: mon.b calling monitor election 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:47.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: monmap epoch 3 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: fsid 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: last_changed 2024-04-24T23:04:41.798376+0000 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: created 2024-04-24T23:02:41.198045+0000 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: min_mon_release 19 (squid) 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: election_strategy: 1 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: fsmap 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: mgrmap e13: x(active, since 58s) 2024-04-24T23:04:47.264 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: overall HEALTH_OK 2024-04-24T23:04:47.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:47.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:47.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:47.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: mon.a calling monitor election 2024-04-24T23:04:47.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: mon.c calling monitor election 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: mon.b calling monitor election 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:47.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:04:47.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: monmap epoch 3 2024-04-24T23:04:47.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: fsid 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:47.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: last_changed 2024-04-24T23:04:41.798376+0000 2024-04-24T23:04:47.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: created 2024-04-24T23:02:41.198045+0000 2024-04-24T23:04:47.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: min_mon_release 19 (squid) 2024-04-24T23:04:47.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: election_strategy: 1 2024-04-24T23:04:47.325 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-04-24T23:04:47.325 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-24T23:04:47.325 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-04-24T23:04:47.325 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: fsmap 2024-04-24T23:04:47.325 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:47.325 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: mgrmap e13: x(active, since 58s) 2024-04-24T23:04:47.325 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: overall HEALTH_OK 2024-04-24T23:04:47.325 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:47.325 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:47.338 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.334+0000 7fecb30ec700 1 -- 172.21.15.183:0/1041082386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c0035c0 msgr2=0x7fec9c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.334+0000 7fecb30ec700 1 --2- 172.21.15.183:0/1041082386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c0035c0 0x7fec9c005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fecac077370 tx=0x7feca000fe60 comp rx=0 tx=0).stop 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.335+0000 7fecb30ec700 1 -- 172.21.15.183:0/1041082386 shutdown_connections 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.335+0000 7fecb30ec700 1 --2- 172.21.15.183:0/1041082386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c0035c0 0x7fec9c005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.335+0000 7fecb30ec700 1 -- 172.21.15.183:0/1041082386 >> 172.21.15.183:0/1041082386 conn(0x7fecac075f90 msgr2=0x7fecac0763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.335+0000 7fecb30ec700 1 -- 172.21.15.183:0/1041082386 shutdown_connections 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.335+0000 7fecb30ec700 1 -- 172.21.15.183:0/1041082386 wait complete. 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.335+0000 7fecb30ec700 1 Processor -- start 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.335+0000 7fecb30ec700 1 -- start start 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb30ec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c0035c0 0x7fecac19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb30ec700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 0x7fecac197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb30ec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecac197c60 0x7fecac198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb30ec700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fecac109c60 con 0x7fecac197c60 2024-04-24T23:04:47.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb30ec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fecac109ae0 con 0x7fec9c0035c0 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb30ec700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fecac109de0 con 0x7fecac19db90 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecabfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 0x7fecac197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecabfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 0x7fecac197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38950/0 (socket says 172.21.15.183:38950) 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecabfff700 1 -- 172.21.15.183:0/796768233 learned_addr learned my addr 172.21.15.183:0/796768233 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb0e88700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c0035c0 0x7fecac19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecabfff700 1 -- 172.21.15.183:0/796768233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 msgr2=0x7fecac197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecabfff700 1 -- 172.21.15.183:0/796768233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 msgr2=0x7fecac197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecabfff700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 0x7fecac197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecabfff700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 0x7fecac197720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb0e88700 1 -- 172.21.15.183:0/796768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c0035c0 msgr2=0x7fecac19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb0e88700 1 -- 172.21.15.183:0/796768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c0035c0 msgr2=0x7fecac19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb0e88700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c0035c0 0x7fecac19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb1689700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecac197c60 0x7fecac198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb0e88700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c0035c0 0x7fecac19d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb1689700 1 -- 172.21.15.183:0/796768233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 msgr2=0x7fecac197720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb1689700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 0x7fecac197720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb1689700 1 -- 172.21.15.183:0/796768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c0035c0 msgr2=0x7fecac19d650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb1689700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c0035c0 0x7fecac19d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.336+0000 7fecb1689700 1 -- 172.21.15.183:0/796768233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac1a4400 con 0x7fecac197c60 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.337+0000 7fecb1689700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecac197c60 0x7fecac198110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feca400bd50 tx=0x7feca400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.337+0000 7feca9ffb700 1 -- 172.21.15.183:0/796768233 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4019070 con 0x7fecac197c60 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.337+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac1a4690 con 0x7fecac197c60 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.337+0000 7feca9ffb700 1 -- 172.21.15.183:0/796768233 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feca400cd10 con 0x7fecac197c60 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.337+0000 7feca9ffb700 1 -- 172.21.15.183:0/796768233 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4012700 con 0x7fecac197c60 2024-04-24T23:04:47.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.337+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac1a4c00 con 0x7fecac197c60 2024-04-24T23:04:47.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.338+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecac06a150 con 0x7fecac197c60 2024-04-24T23:04:47.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.338+0000 7feca9ffb700 1 -- 172.21.15.183:0/796768233 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7feca401c070 con 0x7fecac197c60 2024-04-24T23:04:47.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.338+0000 7feca9ffb700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec90039580 0x7fec9003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.338+0000 7feca9ffb700 1 -- 172.21.15.183:0/796768233 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feca404bce0 con 0x7fecac197c60 2024-04-24T23:04:47.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.339+0000 7fecb0e88700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec90039580 0x7fec9003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.339+0000 7fecb0e88700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec90039580 0x7fec9003ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feca0005b40 tx=0x7feca0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.344+0000 7feca9ffb700 1 -- 172.21.15.183:0/796768233 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feca4007290 con 0x7fecac197c60 2024-04-24T23:04:47.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:47.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: mon.a calling monitor election 2024-04-24T23:04:47.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: mon.c calling monitor election 2024-04-24T23:04:47.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: mon.b calling monitor election 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: monmap epoch 3 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: fsid 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: last_changed 2024-04-24T23:04:41.798376+0000 2024-04-24T23:04:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: created 2024-04-24T23:02:41.198045+0000 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: min_mon_release 19 (squid) 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: election_strategy: 1 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: fsmap 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: mgrmap e13: x(active, since 58s) 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: overall HEALTH_OK 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:47.651 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.650+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fecac02db30 con 0x7fecac197c60 2024-04-24T23:04:47.652 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.651+0000 7feca9ffb700 1 -- 172.21.15.183:0/796768233 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7feca40139e0 con 0x7fecac197c60 2024-04-24T23:04:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-24T23:04:47.652 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:04:41.798376Z","created":"2024-04-24T23:02:41.198045Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec90039580 msgr2=0x7fec9003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec90039580 0x7fec9003ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feca0005b40 tx=0x7feca0005a90 comp rx=0 tx=0).stop 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecac197c60 msgr2=0x7fecac198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecac197c60 0x7fecac198110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feca400bd50 tx=0x7feca400be30 comp rx=0 tx=0).stop 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 shutdown_connections 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec90039580 0x7fec9003ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecac197c60 0x7fecac198110 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c0035c0 0x7fecac19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 --2- 172.21.15.183:0/796768233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecac19db90 0x7fecac197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 >> 172.21.15.183:0/796768233 conn(0x7fecac075f90 msgr2=0x7fecac106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 shutdown_connections 2024-04-24T23:04:47.656 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:47.654+0000 7fecb30ec700 1 -- 172.21.15.183:0/796768233 wait complete. 2024-04-24T23:04:47.657 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2024-04-24T23:04:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:47 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/796768233' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:04:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:47 smithi161 ceph-mon[32709]: from='client.? 172.21.15.183:0/796768233' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:04:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:48.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:48.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:48.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:48.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:47 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/796768233' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:04:48.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:04:48.908 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T23:04:48.908 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T23:04:49.167 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:49.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:49.541 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:49.541 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.541 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.541 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.541 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.543 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:49.543 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:49.543 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:49.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:49.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:49.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:50.270 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:50 smithi102 ceph-mon[29216]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-24T23:04:50.270 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:50 smithi102 ceph-mon[29216]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-24T23:04:50.270 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:50 smithi102 ceph-mon[29216]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-24T23:04:50.271 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:50 smithi102 ceph-mon[29216]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:50.271 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:50 smithi102 ceph-mon[29216]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:50.271 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:50 smithi102 ceph-mon[29216]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:50.271 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:50 smithi102 ceph-mon[29216]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:04:50.271 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:50 smithi102 ceph-mon[29216]: Reconfiguring daemon mon.a on smithi102 2024-04-24T23:04:50.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:50 smithi183 ceph-mon[33137]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-24T23:04:50.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:50 smithi183 ceph-mon[33137]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-24T23:04:50.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:50 smithi183 ceph-mon[33137]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-24T23:04:50.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:50 smithi183 ceph-mon[33137]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:50.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:50 smithi183 ceph-mon[33137]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:50.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:50 smithi183 ceph-mon[33137]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:50.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:50 smithi183 ceph-mon[33137]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:04:50.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:50 smithi183 ceph-mon[33137]: Reconfiguring daemon mon.a on smithi102 2024-04-24T23:04:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:50 smithi161 ceph-mon[32709]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-24T23:04:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:50 smithi161 ceph-mon[32709]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-24T23:04:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:50 smithi161 ceph-mon[32709]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-24T23:04:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:50 smithi161 ceph-mon[32709]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:50 smithi161 ceph-mon[32709]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:50 smithi161 ceph-mon[32709]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:04:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:50 smithi161 ceph-mon[32709]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:04:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:50 smithi161 ceph-mon[32709]: Reconfiguring daemon mon.a on smithi102 2024-04-24T23:04:51.494 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:51 smithi102 ceph-mon[29216]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:51.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:51 smithi183 ceph-mon[33137]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:51 smithi161 ceph-mon[32709]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:51.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.786+0000 7f51cfe68700 1 -- 172.21.15.102:0/3457112281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c81073b0 msgr2=0x7f51c8107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.786+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3457112281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c81073b0 0x7f51c8107790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009990 tx=0x7f51c4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.786+0000 7f51cfe68700 1 -- 172.21.15.102:0/3457112281 shutdown_connections 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.786+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3457112281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c8101220 0x7f51c8104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.786+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3457112281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c81073b0 0x7f51c8107790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.786+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3457112281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c8100780 0x7f51c8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.786+0000 7f51cfe68700 1 -- 172.21.15.102:0/3457112281 >> 172.21.15.102:0/3457112281 conn(0x7f51c8077730 msgr2=0x7f51c80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.786+0000 7f51cfe68700 1 -- 172.21.15.102:0/3457112281 shutdown_connections 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.786+0000 7f51cfe68700 1 -- 172.21.15.102:0/3457112281 wait complete. 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cfe68700 1 Processor -- start 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cfe68700 1 -- start start 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cfe68700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c8100780 0x7f51c819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cfe68700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c8101220 0x7f51c819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cfe68700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c81073b0 0x7f51c81a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cfe68700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8118c80 con 0x7f51c8100780 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cfe68700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8118b00 con 0x7f51c81073b0 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cfe68700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8118e00 con 0x7f51c8101220 2024-04-24T23:04:51.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cdc04700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c8100780 0x7f51c819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cdc04700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c8100780 0x7f51c819f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35912/0 (socket says 172.21.15.102:35912) 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cdc04700 1 -- 172.21.15.102:0/3517047150 learned_addr learned my addr 172.21.15.102:0/3517047150 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cd403700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c8101220 0x7f51c819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51ce405700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c81073b0 0x7f51c81a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.787+0000 7f51cdc04700 1 -- 172.21.15.102:0/3517047150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c8101220 msgr2=0x7f51c819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51cdc04700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c8101220 0x7f51c819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51cdc04700 1 -- 172.21.15.102:0/3517047150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c81073b0 msgr2=0x7f51c81a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51cdc04700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c81073b0 0x7f51c81a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51cdc04700 1 -- 172.21.15.102:0/3517047150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c81a4190 con 0x7f51c8100780 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51cdc04700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c8100780 0x7f51c819f1d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005910 tx=0x7f51c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51beffd700 1 -- 172.21.15.102:0/3517047150 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4027070 con 0x7f51c8100780 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c81a4420 con 0x7f51c8100780 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51beffd700 1 -- 172.21.15.102:0/3517047150 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51c4022470 con 0x7f51c8100780 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51beffd700 1 -- 172.21.15.102:0/3517047150 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c400b610 con 0x7f51c8100780 2024-04-24T23:04:51.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.788+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c81abd00 con 0x7f51c8100780 2024-04-24T23:04:51.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.789+0000 7f51beffd700 1 -- 172.21.15.102:0/3517047150 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f51c402a070 con 0x7f51c8100780 2024-04-24T23:04:51.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.789+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c8101a30 con 0x7f51c8100780 2024-04-24T23:04:51.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.789+0000 7f51beffd700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f51b4039330 0x7f51b403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.789+0000 7f51beffd700 1 -- 172.21.15.102:0/3517047150 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f51c404d2e0 con 0x7f51c8100780 2024-04-24T23:04:51.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.789+0000 7f51cd403700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f51b4039330 0x7f51b403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.790+0000 7f51cd403700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f51b4039330 0x7f51b403b7f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51c81077d0 tx=0x7f51b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:51.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.794+0000 7f51beffd700 1 -- 172.21.15.102:0/3517047150 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f51c4013070 con 0x7f51c8100780 2024-04-24T23:04:51.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.991+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f51c806a780 con 0x7f51c8100780 2024-04-24T23:04:51.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.991+0000 7f51beffd700 1 -- 172.21.15.102:0/3517047150 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f51c40213d0 con 0x7f51c8100780 2024-04-24T23:04:51.993 INFO:teuthology.orchestra.run.smithi102.stdout:# minimal ceph.conf for 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:51.993 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2024-04-24T23:04:51.993 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:51.993 INFO:teuthology.orchestra.run.smithi102.stdout: mon_host = [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-04-24T23:04:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f51b4039330 msgr2=0x7f51b403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f51b4039330 0x7f51b403b7f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51c81077d0 tx=0x7f51b8009260 comp rx=0 tx=0).stop 2024-04-24T23:04:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c8100780 msgr2=0x7f51c819f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c8100780 0x7f51c819f1d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005910 tx=0x7f51c40047b0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 shutdown_connections 2024-04-24T23:04:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c81073b0 0x7f51c81a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f51b4039330 0x7f51b403b7f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51c8100780 0x7f51c819f1d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 --2- 172.21.15.102:0/3517047150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c8101220 0x7f51c819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 >> 172.21.15.102:0/3517047150 conn(0x7f51c8077730 msgr2=0x7f51c8102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:51.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 shutdown_connections 2024-04-24T23:04:51.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:04:51.993+0000 7f51cfe68700 1 -- 172.21.15.102:0/3517047150 wait complete. 2024-04-24T23:04:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:52 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:52 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:52 smithi102 ceph-mon[29216]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:04:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:52 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:52 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:52 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:52 smithi102 ceph-mon[29216]: Reconfiguring daemon mon.b on smithi161 2024-04-24T23:04:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:52 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3517047150' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:52 smithi102 ceph-mon[29216]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:52.678 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T23:04:52.679 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:04:52.679 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:04:52.706 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:04:52.706 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:52.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:52 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:52.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:52 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:52.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:52 smithi183 ceph-mon[33137]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:04:52.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:52 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:52.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:52 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:52.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:52 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:52.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:52 smithi183 ceph-mon[33137]: Reconfiguring daemon mon.b on smithi161 2024-04-24T23:04:52.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:52 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3517047150' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:52.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:52 smithi183 ceph-mon[33137]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:52.770 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T23:04:52.770 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:04:52.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:52 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:52.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:52 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:52.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:52 smithi161 ceph-mon[32709]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:04:52.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:52 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:52.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:52 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:52.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:52 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:52.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:52 smithi161 ceph-mon[32709]: Reconfiguring daemon mon.b on smithi161 2024-04-24T23:04:52.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:52 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3517047150' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:52.793 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:52 smithi161 ceph-mon[32709]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:52.797 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T23:04:52.797 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:52.861 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T23:04:52.861 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:04:52.891 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T23:04:52.891 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:52.959 INFO:tasks.cephadm:Adding mgr.x on smithi102 2024-04-24T23:04:52.959 INFO:tasks.cephadm:Adding mgr.y on smithi161 2024-04-24T23:04:52.959 INFO:tasks.cephadm:Adding mgr.z on smithi183 2024-04-24T23:04:52.960 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi102=x;smithi161=y;smithi183=z' 2024-04-24T23:04:53.267 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:04:54.302 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:54.302 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:54.302 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:54 smithi183 ceph-mon[33137]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:04:54.302 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:54.303 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:54.303 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:54.303 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:54 smithi183 ceph-mon[33137]: Reconfiguring daemon mon.c on smithi183 2024-04-24T23:04:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:54 smithi102 ceph-mon[29216]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:04:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:54 smithi102 ceph-mon[29216]: Reconfiguring daemon mon.c on smithi183 2024-04-24T23:04:54.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:54.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:54.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:54 smithi161 ceph-mon[32709]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:04:54.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:04:54.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:04:54.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:54.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:54 smithi161 ceph-mon[32709]: Reconfiguring daemon mon.c on smithi183 2024-04-24T23:04:54.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.815+0000 7f26b8aaf700 1 -- 172.21.15.183:0/2783953371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 msgr2=0x7f26b41021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.815+0000 7f26b8aaf700 1 --2- 172.21.15.183:0/2783953371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 0x7f26b41021e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009910 tx=0x7f26a4009c20 comp rx=0 tx=0).stop 2024-04-24T23:04:54.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.815+0000 7f26b8aaf700 1 -- 172.21.15.183:0/2783953371 shutdown_connections 2024-04-24T23:04:54.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.815+0000 7f26b8aaf700 1 --2- 172.21.15.183:0/2783953371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 0x7f26b41021e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.815+0000 7f26b8aaf700 1 -- 172.21.15.183:0/2783953371 >> 172.21.15.183:0/2783953371 conn(0x7f26b40fd360 msgr2=0x7f26b40ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:54.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.815+0000 7f26b8aaf700 1 -- 172.21.15.183:0/2783953371 shutdown_connections 2024-04-24T23:04:54.818 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.815+0000 7f26b8aaf700 1 -- 172.21.15.183:0/2783953371 wait complete. 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.816+0000 7f26b8aaf700 1 Processor -- start 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.816+0000 7f26b8aaf700 1 -- start start 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.816+0000 7f26b8aaf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 0x7f26b4197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.816+0000 7f26b8aaf700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4197a40 0x7f26b419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.816+0000 7f26b8aaf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26b4197ec0 0x7f26b419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.816+0000 7f26b8aaf700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4107d00 con 0x7f26b4101dc0 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.816+0000 7f26b8aaf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4107b80 con 0x7f26b4197ec0 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.816+0000 7f26b8aaf700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4107e80 con 0x7f26b4197a40 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 0x7f26b4197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.821 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 0x7f26b4197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.183:36902/0 (socket says 172.21.15.183:36902) 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b259c700 1 -- 172.21.15.183:0/3395529455 learned_addr learned my addr 172.21.15.183:0/3395529455 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26abfff700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4197a40 0x7f26b419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b2d9d700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26b4197ec0 0x7f26b419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b259c700 1 -- 172.21.15.183:0/3395529455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4197a40 msgr2=0x7f26b419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b259c700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4197a40 0x7f26b419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b259c700 1 -- 172.21.15.183:0/3395529455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26b4197ec0 msgr2=0x7f26b419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b259c700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26b4197ec0 0x7f26b419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b259c700 1 -- 172.21.15.183:0/3395529455 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b419f910 con 0x7f26b4101dc0 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b259c700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 0x7f26b4197500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005e00 tx=0x7f26a4004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26ab7fe700 1 -- 172.21.15.183:0/3395529455 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4026070 con 0x7f26b4101dc0 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26ab7fe700 1 -- 172.21.15.183:0/3395529455 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26a400b750 con 0x7f26b4101dc0 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b419fba0 con 0x7f26b4101dc0 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26ab7fe700 1 -- 172.21.15.183:0/3395529455 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a40038e0 con 0x7f26b4101dc0 2024-04-24T23:04:54.822 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.817+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41a0080 con 0x7f26b4101dc0 2024-04-24T23:04:54.823 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.818+0000 7f26ab7fe700 1 -- 172.21.15.183:0/3395529455 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f26a4029070 con 0x7f26b4101dc0 2024-04-24T23:04:54.823 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.818+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b4063890 con 0x7f26b4101dc0 2024-04-24T23:04:54.823 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.818+0000 7f26ab7fe700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f26a0039580 0x7f26a003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.823 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.819+0000 7f26ab7fe700 1 -- 172.21.15.183:0/3395529455 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f26a404ced0 con 0x7f26b4101dc0 2024-04-24T23:04:54.823 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.819+0000 7f26abfff700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f26a0039580 0x7f26a003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.823 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.819+0000 7f26abfff700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f26a0039580 0x7f26a003ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f269c009ad0 tx=0x7f269c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.826 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:54.824+0000 7f26ab7fe700 1 -- 172.21.15.183:0/3395529455 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f26a4013070 con 0x7f26b4101dc0 2024-04-24T23:04:55.041 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.038+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi102=x;smithi161=y;smithi183=z", "target": ["mon-mgr", ""]}) -- 0x7f26b4062670 con 0x7f26a0039580 2024-04-24T23:04:55.049 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.046+0000 7f26ab7fe700 1 -- 172.21.15.183:0/3395529455 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f26b4062670 con 0x7f26a0039580 2024-04-24T23:04:55.049 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2024-04-24T23:04:55.051 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f26a0039580 msgr2=0x7f26a003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:55.051 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f26a0039580 0x7f26a003ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f269c009ad0 tx=0x7f269c009210 comp rx=0 tx=0).stop 2024-04-24T23:04:55.051 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 msgr2=0x7f26b4197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:55.051 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 0x7f26b4197500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005e00 tx=0x7f26a4004f80 comp rx=0 tx=0).stop 2024-04-24T23:04:55.051 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 shutdown_connections 2024-04-24T23:04:55.051 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26b4197ec0 0x7f26b419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.052 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f26a0039580 0x7f26a003ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.052 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26b4101dc0 0x7f26b4197500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.052 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 --2- 172.21.15.183:0/3395529455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4197a40 0x7f26b419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.052 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 >> 172.21.15.183:0/3395529455 conn(0x7f26b40fd360 msgr2=0x7f26b40fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:55.052 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 shutdown_connections 2024-04-24T23:04:55.052 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:04:55.049+0000 7f26b8aaf700 1 -- 172.21.15.183:0/3395529455 wait complete. 2024-04-24T23:04:55.159 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:55 smithi183 ceph-mon[33137]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:55.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:55 smithi102 ceph-mon[29216]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:55.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:55 smithi161 ceph-mon[32709]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:56.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:56 smithi102 ceph-mon[29216]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi102=x;smithi161=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:56.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:56 smithi102 ceph-mon[29216]: Saving service mgr spec with placement smithi102=x;smithi161=y;smithi183=z;count:3 2024-04-24T23:04:56.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:56.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:56 smithi161 ceph-mon[32709]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi102=x;smithi161=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:56.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:56 smithi161 ceph-mon[32709]: Saving service mgr spec with placement smithi102=x;smithi161=y;smithi183=z;count:3 2024-04-24T23:04:56.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:56.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:56 smithi183 ceph-mon[33137]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi102=x;smithi161=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:56.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:56 smithi183 ceph-mon[33137]: Saving service mgr spec with placement smithi102=x;smithi161=y;smithi183=z;count:3 2024-04-24T23:04:56.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:56.423 DEBUG:teuthology.orchestra.run.smithi161:mgr.y> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T23:04:56.426 DEBUG:teuthology.orchestra.run.smithi183:mgr.z> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T23:04:56.428 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T23:04:56.429 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:04:56.429 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:04:56.444 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:04:56.444 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-04-24T23:04:56.499 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:04:56.499 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:56.499 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-24T23:04:56.499 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:56.499 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:56.499 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-24 23:04:01.175624234 +0000 2024-04-24T23:04:56.500 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-24 22:58:46.791254443 +0000 2024-04-24T23:04:56.500 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-24 22:58:46.791254443 +0000 2024-04-24T23:04:56.500 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-24 22:58:46.791254443 +0000 2024-04-24T23:04:56.500 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:04:56.563 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-24T23:04:56.564 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-24T23:04:56.564 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.00019877 s, 2.6 MB/s 2024-04-24T23:04:56.565 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:04:56.620 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-04-24T23:04:56.675 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:04:56.675 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:56.675 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-24T23:04:56.676 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:56.676 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:56.676 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-24 23:04:01.206623611 +0000 2024-04-24T23:04:56.676 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-24 22:58:47.119247754 +0000 2024-04-24T23:04:56.676 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-24 22:58:47.119247754 +0000 2024-04-24T23:04:56.676 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-24 22:58:47.119247754 +0000 2024-04-24T23:04:56.676 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:04:56.738 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-24T23:04:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-24T23:04:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000207767 s, 2.5 MB/s 2024-04-24T23:04:56.740 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:04:56.796 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-04-24T23:04:56.851 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:04:56.851 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:56.851 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-24T23:04:56.851 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:56.851 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:56.851 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-24 23:04:01.235623029 +0000 2024-04-24T23:04:56.851 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-24 22:58:47.451240984 +0000 2024-04-24T23:04:56.852 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-24 22:58:47.451240984 +0000 2024-04-24T23:04:56.852 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-24 22:58:47.451240984 +0000 2024-04-24T23:04:56.852 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:04:56.913 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-24T23:04:56.913 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-24T23:04:56.913 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000196079 s, 2.6 MB/s 2024-04-24T23:04:56.914 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:04:56.970 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-04-24T23:04:57.025 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:04:57.025 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:57.025 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-24T23:04:57.025 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:57.025 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:57.025 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-24 23:04:01.267622387 +0000 2024-04-24T23:04:57.025 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-24 22:58:47.789234092 +0000 2024-04-24T23:04:57.026 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-24 22:58:47.789234092 +0000 2024-04-24T23:04:57.026 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-24 22:58:47.789234092 +0000 2024-04-24T23:04:57.026 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:04:57.087 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-24T23:04:57.087 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-24T23:04:57.087 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000165612 s, 3.1 MB/s 2024-04-24T23:04:57.088 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:04:57.144 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-24T23:04:57.161 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:04:57.180 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:04:57.180 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-04-24T23:04:57.237 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:04:57.238 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:57.238 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T23:04:57.238 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:57.238 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:57.238 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-04-24 23:04:41.211886540 +0000 2024-04-24T23:04:57.238 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-04-24 22:58:53.946756881 +0000 2024-04-24T23:04:57.238 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-04-24 22:58:53.946756881 +0000 2024-04-24T23:04:57.238 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-04-24 22:58:53.946756881 +0000 2024-04-24T23:04:57.238 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:04:57.303 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-04-24T23:04:57.304 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-04-24T23:04:57.304 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000268868 s, 1.9 MB/s 2024-04-24T23:04:57.305 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:04:57.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:57.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:57.366 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-04-24T23:04:57.391 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:57.391 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:57.391 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:57.391 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:57.391 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:57.391 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:04:57.391 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:04:57.391 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:04:57.391 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:04:57.392 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:04:57.392 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:57.424 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:04:57.424 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:57.424 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T23:04:57.424 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:57.424 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:57.424 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-04-24 23:04:41.212886519 +0000 2024-04-24T23:04:57.424 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-04-24 22:58:54.304749334 +0000 2024-04-24T23:04:57.424 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-04-24 22:58:54.304749334 +0000 2024-04-24T23:04:57.424 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-04-24 22:58:54.304749334 +0000 2024-04-24T23:04:57.424 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:04:57.489 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-04-24T23:04:57.489 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-04-24T23:04:57.489 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000220092 s, 2.3 MB/s 2024-04-24T23:04:57.491 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:04:57.548 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-04-24T23:04:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:04:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:57.605 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-24T23:04:57.605 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:57.605 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:57.605 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-04-24 23:04:41.212886519 +0000 2024-04-24T23:04:57.605 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-04-24 22:58:54.656741912 +0000 2024-04-24T23:04:57.605 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-04-24 22:58:54.656741912 +0000 2024-04-24T23:04:57.605 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-04-24 22:58:54.656741912 +0000 2024-04-24T23:04:57.605 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:04:57.669 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-04-24T23:04:57.669 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-04-24T23:04:57.669 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000210827 s, 2.4 MB/s 2024-04-24T23:04:57.670 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:04:57.728 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-04-24T23:04:57.784 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:04:57.784 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:57.784 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-24T23:04:57.784 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:57.784 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:57.785 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-04-24 23:04:41.212886519 +0000 2024-04-24T23:04:57.785 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-04-24 22:58:54.994734786 +0000 2024-04-24T23:04:57.785 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-04-24 22:58:54.994734786 +0000 2024-04-24T23:04:57.785 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-04-24 22:58:54.994734786 +0000 2024-04-24T23:04:57.785 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:04:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-04-24T23:04:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-04-24T23:04:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000318393 s, 1.6 MB/s 2024-04-24T23:04:57.850 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:04:57.907 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-24T23:04:57.907 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:04:57.923 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:04:57.923 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-04-24T23:04:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:04:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:57.978 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T23:04:57.978 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:57.978 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:57.978 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-24 23:04:31.708608953 +0000 2024-04-24T23:04:57.978 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-24 22:58:46.790180379 +0000 2024-04-24T23:04:57.978 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-24 22:58:46.790180379 +0000 2024-04-24T23:04:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-24 22:58:46.790180379 +0000 2024-04-24T23:04:57.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:04:58.042 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-24T23:04:58.042 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-24T23:04:58.042 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000249619 s, 2.1 MB/s 2024-04-24T23:04:58.043 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:04:58.099 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-04-24T23:04:58.153 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:04:58.154 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:58.154 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T23:04:58.154 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:58.154 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:58.154 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-24 23:04:31.708608953 +0000 2024-04-24T23:04:58.154 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-24 22:58:47.140170960 +0000 2024-04-24T23:04:58.154 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-24 22:58:47.140170960 +0000 2024-04-24T23:04:58.155 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-24 22:58:47.140170960 +0000 2024-04-24T23:04:58.155 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:04:58.216 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-24T23:04:58.217 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-24T23:04:58.217 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000250632 s, 2.0 MB/s 2024-04-24T23:04:58.218 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:04:58.274 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-04-24T23:04:58.328 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:04:58.329 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:58.329 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T23:04:58.329 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:58.329 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:58.329 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-24 23:04:31.708608953 +0000 2024-04-24T23:04:58.329 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-24 22:58:47.489161567 +0000 2024-04-24T23:04:58.329 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-24 22:58:47.489161567 +0000 2024-04-24T23:04:58.329 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-24 22:58:47.489161567 +0000 2024-04-24T23:04:58.329 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:04:58.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:58 smithi161 ceph-mon[32709]: Deploying daemon mgr.z on smithi183 2024-04-24T23:04:58.390 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:58 smithi183 ceph-mon[33137]: Deploying daemon mgr.z on smithi183 2024-04-24T23:04:58.390 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-24T23:04:58.390 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-24T23:04:58.391 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000214365 s, 2.4 MB/s 2024-04-24T23:04:58.392 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:04:58.447 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-04-24T23:04:58.502 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:04:58.502 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:04:58.502 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-24T23:04:58.502 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:04:58.502 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:04:58.502 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-24 23:04:31.709608926 +0000 2024-04-24T23:04:58.502 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-24 22:58:47.822152605 +0000 2024-04-24T23:04:58.502 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-24 22:58:47.822152605 +0000 2024-04-24T23:04:58.503 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-24 22:58:47.822152605 +0000 2024-04-24T23:04:58.503 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:04:58.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:58 smithi102 ceph-mon[29216]: Deploying daemon mgr.z on smithi183 2024-04-24T23:04:58.650 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-24T23:04:58.650 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-24T23:04:58.650 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000200842 s, 2.5 MB/s 2024-04-24T23:04:58.651 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:04:58.666 INFO:tasks.cephadm:Deploying osd.0 on smithi102 with /dev/vg_nvme/lv_4... 2024-04-24T23:04:58.667 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:04:58.905 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:04:59.012 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:58 smithi183 systemd[1]: Starting Ceph mgr.z for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:04:59.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:04:59 smithi102 ceph-mon[29216]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:59.326 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:04:59 smithi183 ceph-mon[33137]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:59.326 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 podman[34986]: 2024-04-24 23:04:58.96987381 +0000 UTC m=+0.017703844 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:04:59.326 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 podman[34986]: 2024-04-24 23:04:59.106551371 +0000 UTC m=+0.154381437 container create a929bbfac236eff30661a490426103df687e9f21d8e0650b905c89ea072f7a57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:04:59.326 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 podman[34986]: 2024-04-24 23:04:59.32537368 +0000 UTC m=+0.373203687 container init a929bbfac236eff30661a490426103df687e9f21d8e0650b905c89ea072f7a57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T23:04:59.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:04:59 smithi161 ceph-mon[32709]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:59.666 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 podman[34986]: 2024-04-24 23:04:59.328113359 +0000 UTC m=+0.375943367 container start a929bbfac236eff30661a490426103df687e9f21d8e0650b905c89ea072f7a57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T23:04:59.667 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 ceph-mgr[35005]: -- 172.21.15.183:0/1079955760 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614e485e5a0 con 0x5614e4832800 2024-04-24T23:04:59.667 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 bash[34986]: a929bbfac236eff30661a490426103df687e9f21d8e0650b905c89ea072f7a57 2024-04-24T23:04:59.667 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 systemd[1]: Started Ceph mgr.z for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:04:59.667 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:04:59.480+0000 7fe145c28200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:04:59.667 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:04:59.665+0000 7fe145c28200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:05:00.012 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:04:59 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:04:59.767+0000 7fe145c28200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:05:00.478 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:00.074+0000 7fe145c28200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:05:00.478 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:00.158+0000 7fe145c28200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:05:00.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:00.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:00.596 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:05:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:05:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:05:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:00 smithi161 ceph-mon[32709]: Deploying daemon mgr.y on smithi161 2024-04-24T23:05:00.597 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:00 smithi161 ceph-mon[32709]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:05:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:05:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:05:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-mon[33137]: Deploying daemon mgr.y on smithi161 2024-04-24T23:05:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:00 smithi183 ceph-mon[33137]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:05:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:05:00.763 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-24T23:05:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-24T23:05:00.763 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114335 s, 917 MB/s 2024-04-24T23:05:00.763 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-24T23:05:00.763 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:00.763 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:00.763 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:00.764 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:05:00.764 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:05:00.764 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:05:00.764 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:00.764 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:00 smithi102 ceph-mon[29216]: Deploying daemon mgr.y on smithi161 2024-04-24T23:05:00.764 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:00 smithi102 ceph-mon[29216]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:01.373 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2024-04-24T23:05:01.521 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:01 smithi161 systemd[1]: Starting Ceph mgr.y for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:05:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:05:01.701 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:01 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:01.361+0000 7fe145c28200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:05:01.701 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:01 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:01.529+0000 7fe145c28200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:05:01.702 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:01 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:01.699+0000 7fe145c28200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:05:01.787 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:01 smithi161 podman[34092]: 2024-04-24 23:05:01.423664671 +0000 UTC m=+0.024218007 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:05:01.787 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:01 smithi161 podman[34092]: 2024-04-24 23:05:01.565247735 +0000 UTC m=+0.165801040 container create 0d561e3591389cbb20303ffb1fc5866f12bec25bffbd86b0f27e8f5e6f768624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-24T23:05:01.787 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:01 smithi161 podman[34092]: 2024-04-24 23:05:01.786551008 +0000 UTC m=+0.387104313 container init 0d561e3591389cbb20303ffb1fc5866f12bec25bffbd86b0f27e8f5e6f768624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-24T23:05:02.109 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:01 smithi161 podman[34092]: 2024-04-24 23:05:01.789173299 +0000 UTC m=+0.389726612 container start 0d561e3591389cbb20303ffb1fc5866f12bec25bffbd86b0f27e8f5e6f768624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T23:05:02.109 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:01 smithi161 ceph-mgr[34110]: -- 172.21.15.161:0/913026870 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a9f5b25a0 con 0x563a9f587000 2024-04-24T23:05:02.109 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:01 smithi161 bash[34092]: 0d561e3591389cbb20303ffb1fc5866f12bec25bffbd86b0f27e8f5e6f768624 2024-04-24T23:05:02.109 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:01 smithi161 systemd[1]: Started Ceph mgr.y for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:05:02.109 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:01 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:01.954+0000 7f4de9328200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:05:02.262 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:01 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:01.985+0000 7fe145c28200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:05:02.262 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:02 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:02.078+0000 7fe145c28200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:05:02.589 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:02.146+0000 7f4de9328200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:05:02.589 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:02.257+0000 7f4de9328200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:05:02.589 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:02.586+0000 7f4de9328200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:05:02.762 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:02 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:02.294+0000 7fe145c28200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:05:02.762 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:02 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:02.492+0000 7fe145c28200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:05:02.859 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:02.685+0000 7f4de9328200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:05:03.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.174+0000 7f767b889700 1 -- 172.21.15.102:0/2031956515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 msgr2=0x7f7674101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.174+0000 7f767b889700 1 --2- 172.21.15.102:0/2031956515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 0x7f7674101850 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7670009990 tx=0x7f7670009ca0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.174+0000 7f767b889700 1 -- 172.21.15.102:0/2031956515 shutdown_connections 2024-04-24T23:05:03.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.174+0000 7f767b889700 1 --2- 172.21.15.102:0/2031956515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7674101d90 0x7f7674109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.174+0000 7f767b889700 1 --2- 172.21.15.102:0/2031956515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7674102800 0x7f7674102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.174+0000 7f767b889700 1 --2- 172.21.15.102:0/2031956515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 0x7f7674101850 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.174+0000 7f767b889700 1 -- 172.21.15.102:0/2031956515 >> 172.21.15.102:0/2031956515 conn(0x7f76740fbec0 msgr2=0x7f76740fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:03.177 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.177 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.177 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.177 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.177 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:03.177 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:02 smithi102 ceph-mon[29216]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:03.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.174+0000 7f767b889700 1 -- 172.21.15.102:0/2031956515 shutdown_connections 2024-04-24T23:05:03.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.174+0000 7f767b889700 1 -- 172.21.15.102:0/2031956515 wait complete. 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f767b889700 1 Processor -- start 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f767b889700 1 -- start start 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f767b889700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 0x7f7674100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f767b889700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7674101d90 0x7f767419d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f767b889700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7674102800 0x7f767419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f767b889700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f767410cbf0 con 0x7f76741013d0 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f767b889700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f767410ca70 con 0x7f7674102800 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f767b889700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f767410cd70 con 0x7f7674101d90 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679625700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 0x7f7674100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679625700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 0x7f7674100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39300/0 (socket says 172.21.15.102:39300) 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679625700 1 -- 172.21.15.102:0/864198903 learned_addr learned my addr 172.21.15.102:0/864198903 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7678e24700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7674101d90 0x7f767419d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679e26700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7674102800 0x7f767419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679625700 1 -- 172.21.15.102:0/864198903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7674101d90 msgr2=0x7f767419d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679625700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7674101d90 0x7f767419d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679625700 1 -- 172.21.15.102:0/864198903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7674102800 msgr2=0x7f767419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679625700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7674102800 0x7f767419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679625700 1 -- 172.21.15.102:0/864198903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767419ff90 con 0x7f76741013d0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.175+0000 7f7679625700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 0x7f7674100e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f766400d950 tx=0x7f766400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.176+0000 7f766a7fc700 1 -- 172.21.15.102:0/864198903 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664016070 con 0x7f76741013d0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.176+0000 7f766a7fc700 1 -- 172.21.15.102:0/864198903 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f766400fd40 con 0x7f76741013d0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.176+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7674196ea0 con 0x7f76741013d0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.176+0000 7f766a7fc700 1 -- 172.21.15.102:0/864198903 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766400e3b0 con 0x7f76741013d0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.176+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76741973e0 con 0x7f76741013d0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.177+0000 7f766a7fc700 1 -- 172.21.15.102:0/864198903 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7664019070 con 0x7f76741013d0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.177+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7674068f20 con 0x7f76741013d0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.177+0000 7f766a7fc700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7660039330 0x7f766003b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.177+0000 7f766a7fc700 1 -- 172.21.15.102:0/864198903 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f766404be10 con 0x7f76741013d0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.177+0000 7f7678e24700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7660039330 0x7f766003b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.177+0000 7f7678e24700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7660039330 0x7f766003b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76741013a0 tx=0x7f7670005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:03.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.182+0000 7f766a7fc700 1 -- 172.21.15.102:0/864198903 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f766401a570 con 0x7f76741013d0 2024-04-24T23:05:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:03.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:02 smithi183 ceph-mon[33137]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:03.263 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:03 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:03.149+0000 7fe145c28200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:05:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:02 smithi161 ceph-mon[32709]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:03.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:03.390+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7674062670 con 0x7f7660039330 2024-04-24T23:05:03.897 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:03 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:03.596+0000 7fe145c28200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:05:03.897 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:03 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:03.696+0000 7fe145c28200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:05:03.898 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:03 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:03.896+0000 7fe145c28200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:05:04.202 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:03 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:03.930+0000 7f4de9328200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:05:04.202 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:04.100+0000 7f4de9328200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:04.284 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.285 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:05:04.285 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:05:04.285 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:04.419+0000 7fe145c28200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:04.509+0000 7fe145c28200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:05:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:05:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:04.561 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:04.272+0000 7f4de9328200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:05:04.561 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:04.559+0000 7f4de9328200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:05:04.561 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.561 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.561 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.561 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:04.561 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:04.561 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:04.562 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.562 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.562 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:04.562 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:04.562 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:04.562 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:05:04.562 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:05:04.562 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:04.762 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:04.747+0000 7fe145c28200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:05:04.858 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:04.651+0000 7f4de9328200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:05:04.858 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:04 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:04.856+0000 7f4de9328200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:05:05.109 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:05 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:05.047+0000 7f4de9328200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:05:05.178 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:04.879+0000 7fe145c28200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:05:05.178 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:04 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:04.972+0000 7fe145c28200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:05:05.179 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:05 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:05.087+0000 7fe145c28200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:05:05.179 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:05 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:05.177+0000 7fe145c28200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:05:05.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:05 smithi102 ceph-mon[29216]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:05.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:05 smithi102 ceph-mon[29216]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:05:05.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:05 smithi102 ceph-mon[29216]: Reconfiguring daemon mgr.x on smithi102 2024-04-24T23:05:05.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:05 smithi102 ceph-mon[29216]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:05 smithi183 ceph-mon[33137]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:05 smithi183 ceph-mon[33137]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:05:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:05 smithi183 ceph-mon[33137]: Reconfiguring daemon mgr.x on smithi102 2024-04-24T23:05:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:05 smithi183 ceph-mon[33137]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:05 smithi161 ceph-mon[32709]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:05 smithi161 ceph-mon[32709]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:05:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:05 smithi161 ceph-mon[32709]: Reconfiguring daemon mgr.x on smithi102 2024-04-24T23:05:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:05 smithi161 ceph-mon[32709]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:06.087 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:05 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:05.664+0000 7f4de9328200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:05:06.262 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:05 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:05.990+0000 7fe145c28200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:05:06.262 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:05:06 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:05:06.203+0000 7fe145c28200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:05:06.359 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:06 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:06.085+0000 7f4de9328200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:05:06.359 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:06 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:06.182+0000 7f4de9328200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:05:06.859 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:06 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:06.385+0000 7f4de9328200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:05:07.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:07.153+0000 7f766a7fc700 1 -- 172.21.15.102:0/864198903 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f766400f890 con 0x7f76741013d0 2024-04-24T23:05:07.253 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:06 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:06.918+0000 7f4de9328200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:05:07.253 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:07.009+0000 7f4de9328200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: Standby manager daemon z started 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:05:07.254 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:05:07.255 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:05:07.255 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:05:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: Standby manager daemon z started 2024-04-24T23:05:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:05:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:05:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:05:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:07 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:05:07.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: Standby manager daemon z started 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:05:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:07 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.183:0/505897025' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:05:07.597 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:07.251+0000 7f4de9328200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:05:07.597 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:07.383+0000 7f4de9328200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:05:07.597 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:07.479+0000 7f4de9328200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:05:07.597 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:07.595+0000 7f4de9328200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:05:07.859 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:07 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:07.686+0000 7f4de9328200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:05:08.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:08 smithi161 ceph-mon[32709]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-24T23:05:08.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:05:08.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:08 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3624279975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fabc1206-39d7-4c69-9633-e64bf8677c8c"}]: dispatch 2024-04-24T23:05:08.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:08 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3624279975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fabc1206-39d7-4c69-9633-e64bf8677c8c"}]': finished 2024-04-24T23:05:08.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:08 smithi161 ceph-mon[32709]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:05:08.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:08.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:08 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/493373020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:08.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:08 smithi183 ceph-mon[33137]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-24T23:05:08.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:05:08.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:08 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3624279975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fabc1206-39d7-4c69-9633-e64bf8677c8c"}]: dispatch 2024-04-24T23:05:08.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:08 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3624279975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fabc1206-39d7-4c69-9633-e64bf8677c8c"}]': finished 2024-04-24T23:05:08.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:08 smithi183 ceph-mon[33137]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:05:08.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:08.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:08 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/493373020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:08 smithi102 ceph-mon[29216]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-24T23:05:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:05:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:08 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3624279975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fabc1206-39d7-4c69-9633-e64bf8677c8c"}]: dispatch 2024-04-24T23:05:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:08 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3624279975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fabc1206-39d7-4c69-9633-e64bf8677c8c"}]': finished 2024-04-24T23:05:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:08 smithi102 ceph-mon[29216]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:05:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:08 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/493373020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:08.790 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:08 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:08.498+0000 7f4de9328200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:05:08.790 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:05:08 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:05:08.712+0000 7f4de9328200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:05:09.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:09.246+0000 7f766a7fc700 1 -- 172.21.15.102:0/864198903 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f7664030b70 con 0x7f76741013d0 2024-04-24T23:05:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:09 smithi183 ceph-mon[33137]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:09 smithi183 ceph-mon[33137]: Standby manager daemon y started 2024-04-24T23:05:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:09 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:05:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:09 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:05:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:09 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:05:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:09 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:05:09.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:09 smithi102 ceph-mon[29216]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:09 smithi102 ceph-mon[29216]: Standby manager daemon y started 2024-04-24T23:05:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:09 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:05:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:09 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:05:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:09 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:05:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:09 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:05:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:09 smithi161 ceph-mon[32709]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:09 smithi161 ceph-mon[32709]: Standby manager daemon y started 2024-04-24T23:05:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:09 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:05:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:09 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:05:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:09 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:05:09.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:09 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.161:0/3700156191' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:05:10.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:10 smithi183 ceph-mon[33137]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-24T23:05:10.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:10 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:05:10.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:10 smithi102 ceph-mon[29216]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-24T23:05:10.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:10 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:05:10.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:10 smithi161 ceph-mon[32709]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-24T23:05:10.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:10 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:05:11.460 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:11 smithi102 ceph-mon[29216]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:11 smithi183 ceph-mon[33137]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:11 smithi161 ceph-mon[32709]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:13 smithi102 ceph-mon[29216]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:13 smithi161 ceph-mon[32709]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:13 smithi183 ceph-mon[33137]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:15.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:15 smithi102 ceph-mon[29216]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:15.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:05:15.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:15 smithi161 ceph-mon[32709]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:05:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:15 smithi183 ceph-mon[33137]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:05:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:16.541 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:16 smithi102 ceph-mon[29216]: Deploying daemon osd.0 on smithi102 2024-04-24T23:05:16.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:16 smithi161 ceph-mon[32709]: Deploying daemon osd.0 on smithi102 2024-04-24T23:05:16.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:16 smithi183 ceph-mon[33137]: Deploying daemon osd.0 on smithi102 2024-04-24T23:05:17.318 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:17 smithi102 ceph-mon[29216]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:17 smithi161 ceph-mon[32709]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:17 smithi183 ceph-mon[33137]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:19.381 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:19 smithi102 ceph-mon[29216]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:19 smithi161 ceph-mon[32709]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:19.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:19 smithi183 ceph-mon[33137]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:20 smithi183 ceph-mon[33137]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:21.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:21.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:21.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:21.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:20 smithi102 ceph-mon[29216]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:21.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:21.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:21.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:21.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:20 smithi161 ceph-mon[32709]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:23 smithi183 ceph-mon[33137]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:23.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:23 smithi102 ceph-mon[29216]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:23 smithi161 ceph-mon[32709]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:23.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.648+0000 7f766a7fc700 1 -- 172.21.15.102:0/864198903 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7674062670 con 0x7f7660039330 2024-04-24T23:05:23.650 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 0 on host 'smithi102' 2024-04-24T23:05:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7660039330 msgr2=0x7f766003b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7660039330 0x7f766003b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76741013a0 tx=0x7f7670005fd0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 msgr2=0x7f7674100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 0x7f7674100e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f766400d950 tx=0x7f766400dc60 comp rx=0 tx=0).stop 2024-04-24T23:05:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 shutdown_connections 2024-04-24T23:05:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7674102800 0x7f767419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7660039330 0x7f766003b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76741013d0 0x7f7674100e40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 --2- 172.21.15.102:0/864198903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7674101d90 0x7f767419d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 >> 172.21.15.102:0/864198903 conn(0x7f76740fbec0 msgr2=0x7f7674105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:23.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 shutdown_connections 2024-04-24T23:05:23.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:05:23.651+0000 7f767b889700 1 -- 172.21.15.102:0/864198903 wait complete. 2024-04-24T23:05:24.377 DEBUG:teuthology.orchestra.run.smithi102:osd.0> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T23:05:24.379 INFO:tasks.cephadm:Deploying osd.1 on smithi161 with /dev/vg_nvme/lv_4... 2024-04-24T23:05:24.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:05:24.623 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:05:24.745 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:24.745 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:24.745 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:24.745 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:24 smithi161 ceph-mon[32709]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:05:24.745 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:24 smithi161 ceph-mon[32709]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:24.912 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:24.912 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:24.912 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:24.912 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:24 smithi102 ceph-mon[29216]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:05:24.913 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:24 smithi102 ceph-mon[29216]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:25.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:25.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:25.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:25.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:24 smithi183 ceph-mon[33137]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:05:25.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:24 smithi183 ceph-mon[33137]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:25.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:25 smithi102 ceph-mon[29216]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:05:25.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:25 smithi102 ceph-mon[29216]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:05:25.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:25 smithi102 ceph-mon[29216]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:05:25.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:25 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:25.823 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:05:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0[44288]: 2024-04-24T23:05:25.625+0000 7f31e4491700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T23:05:25.823 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:05:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0[44288]: 2024-04-24T23:05:25.629+0000 7f31e0284700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:05:25.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:25 smithi161 ceph-mon[32709]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:05:25.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:25 smithi161 ceph-mon[32709]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:05:25.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:25 smithi161 ceph-mon[32709]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:05:25.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:25 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:26.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:25 smithi183 ceph-mon[33137]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:05:26.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:25 smithi183 ceph-mon[33137]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:05:26.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:25 smithi183 ceph-mon[33137]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:05:26.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:25 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:26.484 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:05:26.484 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:05:26.484 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-04-24T23:05:26.484 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-04-24T23:05:26.484 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123064 s, 852 MB/s 2024-04-24T23:05:26.484 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-04-24T23:05:27.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:26 smithi183 ceph-mon[33137]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:05:27.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:26 smithi183 ceph-mon[33137]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:05:27.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:26 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:27.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:26 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:27.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:26 smithi183 ceph-mon[33137]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:27.048 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:26 smithi102 ceph-mon[29216]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:05:27.048 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:26 smithi102 ceph-mon[29216]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:05:27.048 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:26 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:27.049 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:26 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:27.049 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:26 smithi102 ceph-mon[29216]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:27.095 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:26 smithi161 ceph-mon[32709]: from='osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:05:27.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:26 smithi161 ceph-mon[32709]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:05:27.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:26 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:27.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:26 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:27.096 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:26 smithi161 ceph-mon[32709]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:27.124 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-04-24T23:05:27.363 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:05:27.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:05:27.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:05:27.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] boot 2024-04-24T23:05:27.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:05:27.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:27.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.816 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:27.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:27.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:27.817 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] boot 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.913 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: osd.0 [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] boot 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:27.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:28.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:28 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi102 2024-04-24T23:05:28.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:28 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi102 to 17088M 2024-04-24T23:05:28.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:28 smithi161 ceph-mon[32709]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:05:28.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:28 smithi161 ceph-mon[32709]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:28.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.906+0000 7fa43fdf7700 1 -- 172.21.15.161:0/4261376841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438107f70 msgr2=0x7fa43810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:28.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.906+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/4261376841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438107f70 0x7fa43810a3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa43000b0a0 tx=0x7fa43000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.906+0000 7fa43fdf7700 1 -- 172.21.15.161:0/4261376841 shutdown_connections 2024-04-24T23:05:28.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.906+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/4261376841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438107f70 0x7fa43810a3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.906+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/4261376841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438102d10 0x7fa438105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.906+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/4261376841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438105640 0x7fa438107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.906+0000 7fa43fdf7700 1 -- 172.21.15.161:0/4261376841 >> 172.21.15.161:0/4261376841 conn(0x7fa4380fc720 msgr2=0x7fa4380feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.906+0000 7fa43fdf7700 1 -- 172.21.15.161:0/4261376841 shutdown_connections 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.906+0000 7fa43fdf7700 1 -- 172.21.15.161:0/4261376841 wait complete. 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43fdf7700 1 Processor -- start 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43fdf7700 1 -- start start 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43fdf7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438102d10 0x7fa438198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43fdf7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438105640 0x7fa438198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43fdf7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438107f70 0x7fa43819f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43fdf7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa43810dd80 con 0x7fa438107f70 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43fdf7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa43810dc00 con 0x7fa438102d10 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43fdf7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa43810df00 con 0x7fa438105640 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43db93700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438102d10 0x7fa438198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43db93700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438102d10 0x7fa438198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46544/0 (socket says 172.21.15.161:46544) 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43db93700 1 -- 172.21.15.161:0/1188556986 learned_addr learned my addr 172.21.15.161:0/1188556986 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43d392700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438105640 0x7fa438198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43e394700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438107f70 0x7fa43819f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43db93700 1 -- 172.21.15.161:0/1188556986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438105640 msgr2=0x7fa438198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43db93700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438105640 0x7fa438198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43db93700 1 -- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438107f70 msgr2=0x7fa43819f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43db93700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438107f70 0x7fa43819f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43db93700 1 -- 172.21.15.161:0/1188556986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4381a0080 con 0x7fa438102d10 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43e394700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438107f70 0x7fa43819f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa43db93700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438102d10 0x7fa438198420 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa43400b530 tx=0x7fa43400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.907+0000 7fa42effd700 1 -- 172.21.15.161:0/1188556986 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434015020 con 0x7fa438102d10 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.908+0000 7fa42effd700 1 -- 172.21.15.161:0/1188556986 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa434011ce0 con 0x7fa438102d10 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.908+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4381a0370 con 0x7fa438102d10 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.908+0000 7fa42effd700 1 -- 172.21.15.161:0/1188556986 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43400f2e0 con 0x7fa438102d10 2024-04-24T23:05:28.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.908+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4381a0980 con 0x7fa438102d10 2024-04-24T23:05:28.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.908+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa438068f20 con 0x7fa438102d10 2024-04-24T23:05:28.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.910+0000 7fa42effd700 1 -- 172.21.15.161:0/1188556986 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fa434018020 con 0x7fa438102d10 2024-04-24T23:05:28.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.910+0000 7fa42effd700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4240a2c10 0x7fa4240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:28.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.910+0000 7fa42effd700 1 -- 172.21.15.161:0/1188556986 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa4340cd2b0 con 0x7fa438102d10 2024-04-24T23:05:28.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.910+0000 7fa43d392700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4240a2c10 0x7fa4240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:28.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.911+0000 7fa43d392700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4240a2c10 0x7fa4240a50d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa428009c60 tx=0x7fa428009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:28.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:28.914+0000 7fa42effd700 1 -- 172.21.15.161:0/1188556986 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa43401f020 con 0x7fa438102d10 2024-04-24T23:05:29.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:28 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi102 2024-04-24T23:05:29.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:28 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi102 to 17088M 2024-04-24T23:05:29.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:28 smithi183 ceph-mon[33137]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:05:29.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:28 smithi183 ceph-mon[33137]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:29.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:28 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi102 2024-04-24T23:05:29.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:28 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi102 to 17088M 2024-04-24T23:05:29.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:28 smithi102 ceph-mon[29216]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:05:29.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:28 smithi102 ceph-mon[29216]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:29.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:29.114+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa438062670 con 0x7fa4240a2c10 2024-04-24T23:05:29.893 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:29 smithi161 ceph-mon[32709]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:29.893 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:29.893 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:29.893 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:30.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:29 smithi183 ceph-mon[33137]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:30.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:30.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:30.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:30.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:29 smithi102 ceph-mon[29216]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:30.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:30.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:30.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:30.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:30 smithi161 ceph-mon[32709]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:31.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:30 smithi183 ceph-mon[33137]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:31.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:30 smithi102 ceph-mon[29216]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:31.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:31 smithi161 ceph-mon[32709]: from='client.? 172.21.15.161:0/1877236240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbfe050-2056-4fe6-91d6-ab6f6e9702b0"}]: dispatch 2024-04-24T23:05:31.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:31 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbfe050-2056-4fe6-91d6-ab6f6e9702b0"}]: dispatch 2024-04-24T23:05:31.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:31 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fbfe050-2056-4fe6-91d6-ab6f6e9702b0"}]': finished 2024-04-24T23:05:31.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:31 smithi161 ceph-mon[32709]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:05:31.966 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:31 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:32.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:31 smithi183 ceph-mon[33137]: from='client.? 172.21.15.161:0/1877236240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbfe050-2056-4fe6-91d6-ab6f6e9702b0"}]: dispatch 2024-04-24T23:05:32.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:31 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbfe050-2056-4fe6-91d6-ab6f6e9702b0"}]: dispatch 2024-04-24T23:05:32.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:31 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fbfe050-2056-4fe6-91d6-ab6f6e9702b0"}]': finished 2024-04-24T23:05:32.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:31 smithi183 ceph-mon[33137]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:05:32.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:31 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:32.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:31 smithi102 ceph-mon[29216]: from='client.? 172.21.15.161:0/1877236240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbfe050-2056-4fe6-91d6-ab6f6e9702b0"}]: dispatch 2024-04-24T23:05:32.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:31 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fbfe050-2056-4fe6-91d6-ab6f6e9702b0"}]: dispatch 2024-04-24T23:05:32.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:31 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fbfe050-2056-4fe6-91d6-ab6f6e9702b0"}]': finished 2024-04-24T23:05:32.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:31 smithi102 ceph-mon[29216]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:05:32.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:31 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:32 smithi183 ceph-mon[33137]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:32 smithi183 ceph-mon[33137]: from='client.? 172.21.15.161:0/61644513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:33.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:32 smithi102 ceph-mon[29216]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:33.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:32 smithi102 ceph-mon[29216]: from='client.? 172.21.15.161:0/61644513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:33.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:32 smithi161 ceph-mon[32709]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:33.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:32 smithi161 ceph-mon[32709]: from='client.? 172.21.15.161:0/61644513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:35.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:35 smithi183 ceph-mon[33137]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:35.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:35 smithi161 ceph-mon[32709]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:35.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:35 smithi102 ceph-mon[29216]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:37 smithi183 ceph-mon[33137]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:37.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:37 smithi102 ceph-mon[29216]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:37.591 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:37 smithi161 ceph-mon[32709]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:39.429 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:39 smithi161 ceph-mon[32709]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:39.429 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:05:39.429 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:39.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:39 smithi183 ceph-mon[33137]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:39.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:05:39.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:39.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:39 smithi102 ceph-mon[29216]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:39.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:05:39.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:40.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:40 smithi183 ceph-mon[33137]: Deploying daemon osd.1 on smithi161 2024-04-24T23:05:40.522 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:40 smithi161 ceph-mon[32709]: Deploying daemon osd.1 on smithi161 2024-04-24T23:05:40.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:40 smithi102 ceph-mon[29216]: Deploying daemon osd.1 on smithi161 2024-04-24T23:05:41.317 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:41 smithi161 ceph-mon[32709]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:41 smithi183 ceph-mon[33137]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:41.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:41 smithi102 ceph-mon[29216]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:43.316 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:43 smithi161 ceph-mon[32709]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:43.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:43 smithi183 ceph-mon[33137]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:43.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:43 smithi102 ceph-mon[29216]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:45.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:45.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:45.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:45.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:44 smithi183 ceph-mon[33137]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:45.300 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:45.300 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:45.300 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:45.300 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:44 smithi161 ceph-mon[32709]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:45.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:45.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:45.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:45.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:44 smithi102 ceph-mon[29216]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:47.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.276+0000 7fa42effd700 1 -- 172.21.15.161:0/1188556986 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa438062670 con 0x7fa4240a2c10 2024-04-24T23:05:47.279 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 1 on host 'smithi161' 2024-04-24T23:05:47.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4240a2c10 msgr2=0x7fa4240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:47.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4240a2c10 0x7fa4240a50d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa428009c60 tx=0x7fa428009260 comp rx=0 tx=0).stop 2024-04-24T23:05:47.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438102d10 msgr2=0x7fa438198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:47.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438102d10 0x7fa438198420 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa43400b530 tx=0x7fa43400b840 comp rx=0 tx=0).stop 2024-04-24T23:05:47.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 shutdown_connections 2024-04-24T23:05:47.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4240a2c10 0x7fa4240a50d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:47.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438107f70 0x7fa43819f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:47.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa438102d10 0x7fa438198420 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:47.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 --2- 172.21.15.161:0/1188556986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438105640 0x7fa438198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:47.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 >> 172.21.15.161:0/1188556986 conn(0x7fa4380fc720 msgr2=0x7fa4381035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:47.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.279+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 shutdown_connections 2024-04-24T23:05:47.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:05:47.280+0000 7fa43fdf7700 1 -- 172.21.15.161:0/1188556986 wait complete. 2024-04-24T23:05:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:47 smithi183 ceph-mon[33137]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:47.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:47 smithi102 ceph-mon[29216]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:47.578 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:47 smithi161 ceph-mon[32709]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:47.578 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:47.578 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:47.578 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:47.578 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:47.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:47.896 DEBUG:teuthology.orchestra.run.smithi161:osd.1> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T23:05:47.898 INFO:tasks.cephadm:Deploying osd.2 on smithi183 with /dev/vg_nvme/lv_4... 2024-04-24T23:05:47.898 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:05:48.142 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:05:48.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:48.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:48.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:48.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:48 smithi183 ceph-mon[33137]: from='osd.1 [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:48.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:48 smithi183 ceph-mon[33137]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:48.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:48 smithi102 ceph-mon[29216]: from='osd.1 [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:48 smithi102 ceph-mon[29216]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:48 smithi161 ceph-mon[32709]: from='osd.1 [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:48 smithi161 ceph-mon[32709]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:05:49.404 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:49 smithi183 ceph-mon[33137]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:49.404 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:49 smithi183 ceph-mon[33137]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:05:49.404 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:49 smithi183 ceph-mon[33137]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:05:49.405 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:49.405 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:49 smithi183 ceph-mon[33137]: from='osd.1 [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:05:49.405 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:49 smithi183 ceph-mon[33137]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:05:49.555 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:49 smithi161 ceph-mon[32709]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:49.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:49 smithi161 ceph-mon[32709]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:05:49.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:49 smithi161 ceph-mon[32709]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:05:49.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:49.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:49 smithi161 ceph-mon[32709]: from='osd.1 [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:05:49.556 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:49 smithi161 ceph-mon[32709]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:05:49.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:49 smithi102 ceph-mon[29216]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:49.575 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:49 smithi102 ceph-mon[29216]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:05:49.575 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:49 smithi102 ceph-mon[29216]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:05:49.575 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:49.575 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:49 smithi102 ceph-mon[29216]: from='osd.1 [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:05:49.575 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:49 smithi102 ceph-mon[29216]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:05:49.861 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:05:49 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1[37270]: 2024-04-24T23:05:49.738+0000 7fbd8a5a9700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T23:05:49.861 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:05:49 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1[37270]: 2024-04-24T23:05:49.741+0000 7fbd82394700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:05:49.959 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:05:49.959 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:05:49.959 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-24T23:05:49.960 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-24T23:05:49.960 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111395 s, 941 MB/s 2024-04-24T23:05:49.960 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-24T23:05:50.481 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-04-24T23:05:50.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:50 smithi161 ceph-mon[32709]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:05:50.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:50 smithi161 ceph-mon[32709]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:05:50.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:50.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:50.559 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:50 smithi161 ceph-mon[32709]: from='osd.1 ' entity='osd.1' 2024-04-24T23:05:50.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:50 smithi102 ceph-mon[29216]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:05:50.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:50 smithi102 ceph-mon[29216]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:05:50.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:50.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:50.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:50 smithi102 ceph-mon[29216]: from='osd.1 ' entity='osd.1' 2024-04-24T23:05:50.719 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:05:50.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:50 smithi183 ceph-mon[33137]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:05:50.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:50 smithi183 ceph-mon[33137]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:05:50.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:50.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:50.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:50 smithi183 ceph-mon[33137]: from='osd.1 ' entity='osd.1' 2024-04-24T23:05:51.435 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:05:51.435 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:05:51.435 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: osd.1 [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] boot 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:51.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:51.437 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:05:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:05:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: osd.1 [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] boot 2024-04-24T23:05:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:05:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:51.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:51.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:51.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:05:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:05:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:05:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: osd.1 [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] boot 2024-04-24T23:05:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:05:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:05:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:05:52.347 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:52 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi161 2024-04-24T23:05:52.347 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:52 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi161 to 17088M 2024-04-24T23:05:52.347 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:52 smithi183 ceph-mon[33137]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:05:52.368 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.366+0000 7f196eb4b700 1 -- 172.21.15.183:0/2346018280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968102390 msgr2=0x7f19681027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:52.368 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.366+0000 7f196eb4b700 1 --2- 172.21.15.183:0/2346018280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968102390 0x7f19681027f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1954009990 tx=0x7f1954009ca0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.368 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.366+0000 7f196eb4b700 1 -- 172.21.15.183:0/2346018280 shutdown_connections 2024-04-24T23:05:52.368 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.366+0000 7f196eb4b700 1 --2- 172.21.15.183:0/2346018280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968102d30 0x7f19681095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.368 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.366+0000 7f196eb4b700 1 --2- 172.21.15.183:0/2346018280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1968101190 0x7f19681015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.368 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.366+0000 7f196eb4b700 1 --2- 172.21.15.183:0/2346018280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968102390 0x7f19681027f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.368 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.366+0000 7f196eb4b700 1 -- 172.21.15.183:0/2346018280 >> 172.21.15.183:0/2346018280 conn(0x7f19680fc730 msgr2=0x7f19680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:52.368 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.366+0000 7f196eb4b700 1 -- 172.21.15.183:0/2346018280 shutdown_connections 2024-04-24T23:05:52.368 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.366+0000 7f196eb4b700 1 -- 172.21.15.183:0/2346018280 wait complete. 2024-04-24T23:05:52.370 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196eb4b700 1 Processor -- start 2024-04-24T23:05:52.370 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196eb4b700 1 -- start start 2024-04-24T23:05:52.370 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196eb4b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101190 0x7f1968197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196eb4b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1968102390 0x7f19681978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196eb4b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968102d30 0x7f196819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196eb4b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f196810cca0 con 0x7f1968102390 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196eb4b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f196810cb20 con 0x7f1968102d30 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196eb4b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f196810ce20 con 0x7f1968101190 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196c8e7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101190 0x7f1968197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196c8e7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101190 0x7f1968197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38734/0 (socket says 172.21.15.183:38734) 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.367+0000 7f196c8e7700 1 -- 172.21.15.183:0/482492821 learned_addr learned my addr 172.21.15.183:0/482492821 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f1967fff700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1968102390 0x7f19681978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196d0e8700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968102d30 0x7f196819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196c8e7700 1 -- 172.21.15.183:0/482492821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968102d30 msgr2=0x7f196819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196c8e7700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968102d30 0x7f196819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196c8e7700 1 -- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1968102390 msgr2=0x7f19681978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:52.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196c8e7700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1968102390 0x7f19681978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196c8e7700 1 -- 172.21.15.183:0/482492821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196819efd0 con 0x7f1968101190 2024-04-24T23:05:52.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196d0e8700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968102d30 0x7f196819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:05:52.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196c8e7700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101190 0x7f1968197370 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f195c00b530 tx=0x7f195c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:52.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f1965ffb700 1 -- 172.21.15.183:0/482492821 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c015020 con 0x7f1968101190 2024-04-24T23:05:52.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f1965ffb700 1 -- 172.21.15.183:0/482492821 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f195c011ce0 con 0x7f1968101190 2024-04-24T23:05:52.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196819f2c0 con 0x7f1968101190 2024-04-24T23:05:52.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f1965ffb700 1 -- 172.21.15.183:0/482492821 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c010d60 con 0x7f1968101190 2024-04-24T23:05:52.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.368+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196819f8d0 con 0x7f1968101190 2024-04-24T23:05:52.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.369+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1968068f20 con 0x7f1968101190 2024-04-24T23:05:52.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.370+0000 7f1965ffb700 1 -- 172.21.15.183:0/482492821 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f195c018020 con 0x7f1968101190 2024-04-24T23:05:52.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.371+0000 7f1965ffb700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19580a2c10 0x7f19580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:52.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.371+0000 7f1965ffb700 1 -- 172.21.15.183:0/482492821 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f195c0cced0 con 0x7f1968101190 2024-04-24T23:05:52.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.371+0000 7f1967fff700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19580a2c10 0x7f19580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:52.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.371+0000 7f1967fff700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19580a2c10 0x7f19580a50d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f195400b5c0 tx=0x7f1954005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:52.377 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.375+0000 7f1965ffb700 1 -- 172.21.15.183:0/482492821 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f195c01f020 con 0x7f1968101190 2024-04-24T23:05:52.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:52 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi161 2024-04-24T23:05:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:52 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi161 to 17088M 2024-04-24T23:05:52.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:52 smithi102 ceph-mon[29216]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:05:52.579 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:05:52.577+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1968062670 con 0x7f19580a2c10 2024-04-24T23:05:52.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:52 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi161 2024-04-24T23:05:52.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:52 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi161 to 17088M 2024-04-24T23:05:52.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:52 smithi161 ceph-mon[32709]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:05:53.500 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:53 smithi183 ceph-mon[33137]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:53.501 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:53.501 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:53.501 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:53.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:53 smithi102 ceph-mon[29216]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:53 smithi161 ceph-mon[32709]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:05:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:05:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:54.506 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:54 smithi183 ceph-mon[33137]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:54.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:54 smithi102 ceph-mon[29216]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:54.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:54 smithi161 ceph-mon[32709]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:55.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:55 smithi102 ceph-mon[29216]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:55 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/1930380307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "673517e2-faa1-4141-9f1c-bb11f16713d1"}]: dispatch 2024-04-24T23:05:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:55 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "673517e2-faa1-4141-9f1c-bb11f16713d1"}]: dispatch 2024-04-24T23:05:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:55 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "673517e2-faa1-4141-9f1c-bb11f16713d1"}]': finished 2024-04-24T23:05:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:55 smithi102 ceph-mon[29216]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:05:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:55 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:55 smithi161 ceph-mon[32709]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:55 smithi161 ceph-mon[32709]: from='client.? 172.21.15.183:0/1930380307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "673517e2-faa1-4141-9f1c-bb11f16713d1"}]: dispatch 2024-04-24T23:05:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:55 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "673517e2-faa1-4141-9f1c-bb11f16713d1"}]: dispatch 2024-04-24T23:05:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:55 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "673517e2-faa1-4141-9f1c-bb11f16713d1"}]': finished 2024-04-24T23:05:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:55 smithi161 ceph-mon[32709]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:05:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:55 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:55 smithi183 ceph-mon[33137]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:55 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/1930380307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "673517e2-faa1-4141-9f1c-bb11f16713d1"}]: dispatch 2024-04-24T23:05:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:55 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "673517e2-faa1-4141-9f1c-bb11f16713d1"}]: dispatch 2024-04-24T23:05:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:55 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "673517e2-faa1-4141-9f1c-bb11f16713d1"}]': finished 2024-04-24T23:05:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:55 smithi183 ceph-mon[33137]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:05:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:55 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:05:56.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:56 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/3464309720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:56.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:56 smithi161 ceph-mon[32709]: from='client.? 172.21.15.183:0/3464309720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:56.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:56 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/3464309720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:05:57.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:57 smithi102 ceph-mon[29216]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:57 smithi161 ceph-mon[32709]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:57.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:57 smithi183 ceph-mon[33137]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:58.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:05:58 smithi161 ceph-mon[32709]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:58.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:05:58 smithi183 ceph-mon[33137]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:05:58.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:05:58 smithi102 ceph-mon[29216]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:01 smithi183 ceph-mon[33137]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:01.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:01 smithi102 ceph-mon[29216]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:01 smithi161 ceph-mon[32709]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:03 smithi183 ceph-mon[33137]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:03 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:06:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:03 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:03 smithi183 ceph-mon[33137]: Deploying daemon osd.2 on smithi183 2024-04-24T23:06:03.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:03 smithi102 ceph-mon[29216]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:03 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:06:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:03 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:03 smithi102 ceph-mon[29216]: Deploying daemon osd.2 on smithi183 2024-04-24T23:06:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:03 smithi161 ceph-mon[32709]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:03 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:06:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:03 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:03 smithi161 ceph-mon[32709]: Deploying daemon osd.2 on smithi183 2024-04-24T23:06:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:05 smithi183 ceph-mon[33137]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:05.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:05 smithi102 ceph-mon[29216]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:05 smithi161 ceph-mon[32709]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:07.505 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:07 smithi183 ceph-mon[33137]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:07.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:07 smithi102 ceph-mon[29216]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:07 smithi161 ceph-mon[32709]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:08.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:08 smithi102 ceph-mon[29216]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:08 smithi161 ceph-mon[32709]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:08 smithi183 ceph-mon[33137]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:10.843 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.841+0000 7f1965ffb700 1 -- 172.21.15.183:0/482492821 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1968062670 con 0x7f19580a2c10 2024-04-24T23:06:10.843 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 2 on host 'smithi183' 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19580a2c10 msgr2=0x7f19580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19580a2c10 0x7f19580a50d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f195400b5c0 tx=0x7f1954005fd0 comp rx=0 tx=0).stop 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101190 msgr2=0x7f1968197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101190 0x7f1968197370 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f195c00b530 tx=0x7f195c00b840 comp rx=0 tx=0).stop 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 shutdown_connections 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968102d30 0x7f196819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1968101190 0x7f1968197370 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19580a2c10 0x7f19580a50d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 --2- 172.21.15.183:0/482492821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1968102390 0x7f19681978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 >> 172.21.15.183:0/482492821 conn(0x7f19680fc730 msgr2=0x7f19681057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 shutdown_connections 2024-04-24T23:06:10.845 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:06:10.842+0000 7f196eb4b700 1 -- 172.21.15.183:0/482492821 wait complete. 2024-04-24T23:06:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:11 smithi183 ceph-mon[33137]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:11.865 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:11.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:11.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:11.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:11 smithi161 ceph-mon[32709]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:11.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:11.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:11.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.867 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:11 smithi102 ceph-mon[29216]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:11.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:11.869 DEBUG:teuthology.orchestra.run.smithi183:osd.2> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T23:06:11.879 INFO:tasks.cephadm:Deploying osd.3 on smithi102 with /dev/vg_nvme/lv_3... 2024-04-24T23:06:11.879 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:06:12.130 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:06:12.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:12 smithi183 ceph-mon[33137]: from='osd.2 [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:06:12.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:12 smithi183 ceph-mon[33137]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:06:12.518 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:12 smithi102 ceph-mon[29216]: from='osd.2 [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:06:12.519 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:12 smithi102 ceph-mon[29216]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:06:12.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:12 smithi161 ceph-mon[32709]: from='osd.2 [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:06:12.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:12 smithi161 ceph-mon[32709]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:06:13.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:13 smithi102 ceph-mon[29216]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:13.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:13 smithi102 ceph-mon[29216]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:06:13.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:13 smithi102 ceph-mon[29216]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:06:13.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:13 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:13.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:13 smithi102 ceph-mon[29216]: from='osd.2 [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:06:13.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:13 smithi102 ceph-mon[29216]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:06:13.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:13 smithi183 ceph-mon[33137]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:13.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:13 smithi183 ceph-mon[33137]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:06:13.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:13 smithi183 ceph-mon[33137]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:06:13.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:13 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:13.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:13 smithi183 ceph-mon[33137]: from='osd.2 [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:06:13.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:13 smithi183 ceph-mon[33137]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:06:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:13 smithi161 ceph-mon[32709]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:13 smithi161 ceph-mon[32709]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:06:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:13 smithi161 ceph-mon[32709]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:06:13.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:13 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:13.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:13 smithi161 ceph-mon[32709]: from='osd.2 [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:06:13.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:13 smithi161 ceph-mon[32709]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:06:14.012 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:13 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2[38275]: 2024-04-24T23:06:13.720+0000 7f6024271700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T23:06:14.012 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:13 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2[38275]: 2024-04-24T23:06:13.724+0000 7f601c05c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:06:14.305 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:06:14.305 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:06:14.305 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-24T23:06:14.306 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-24T23:06:14.309 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113964 s, 920 MB/s 2024-04-24T23:06:14.309 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-24T23:06:14.332 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:14 smithi102 ceph-mon[29216]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:06:14.332 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:14 smithi102 ceph-mon[29216]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:06:14.332 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:14 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:14.333 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:14 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:14.333 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:14 smithi102 ceph-mon[29216]: from='osd.2 ' entity='osd.2' 2024-04-24T23:06:14.333 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:14 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:14.506 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:14 smithi183 ceph-mon[33137]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:06:14.506 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:14 smithi183 ceph-mon[33137]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:06:14.506 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:14 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:14.507 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:14 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:14.507 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:14 smithi183 ceph-mon[33137]: from='osd.2 ' entity='osd.2' 2024-04-24T23:06:14.507 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:14 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:14.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:14 smithi161 ceph-mon[32709]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:06:14.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:14 smithi161 ceph-mon[32709]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:06:14.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:14 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:14.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:14 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:14.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:14 smithi161 ceph-mon[32709]: from='osd.2 ' entity='osd.2' 2024-04-24T23:06:14.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:14 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:14.980 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2024-04-24T23:06:15.221 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:06:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:06:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:06:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: osd.2 [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] boot 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:06:15.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:06:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:06:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:15.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:15.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: osd.2 [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] boot 2024-04-24T23:06:15.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:06:15.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:06:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:06:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: osd.2 [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] boot 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:06:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:06:16.653 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:16 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi183 2024-04-24T23:06:16.653 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:16 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-24T23:06:16.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:16 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi183 2024-04-24T23:06:16.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:16 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-24T23:06:16.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:16 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi183 2024-04-24T23:06:16.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:16 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-24T23:06:16.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.882+0000 7fc7b3f46700 1 -- 172.21.15.102:0/1313997793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 msgr2=0x7fc7ac1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.882+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/1313997793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 0x7fc7ac1095e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b0d0 tx=0x7fc7a400b3e0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.882+0000 7fc7b3f46700 1 -- 172.21.15.102:0/1313997793 shutdown_connections 2024-04-24T23:06:16.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.882+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/1313997793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 0x7fc7ac1095e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.882+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/1313997793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7ac101190 0x7fc7ac1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.882+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/1313997793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac102390 0x7fc7ac1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.882+0000 7fc7b3f46700 1 -- 172.21.15.102:0/1313997793 >> 172.21.15.102:0/1313997793 conn(0x7fc7ac0fc730 msgr2=0x7fc7ac0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:16.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.882+0000 7fc7b3f46700 1 -- 172.21.15.102:0/1313997793 shutdown_connections 2024-04-24T23:06:16.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.882+0000 7fc7b3f46700 1 -- 172.21.15.102:0/1313997793 wait complete. 2024-04-24T23:06:16.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b3f46700 1 Processor -- start 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b3f46700 1 -- start start 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b3f46700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101190 0x7fc7ac198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b3f46700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7ac102390 0x7fc7ac198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b3f46700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 0x7fc7ac19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b3f46700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac10cc10 con 0x7fc7ac102d30 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b3f46700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac10ca90 con 0x7fc7ac102390 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b3f46700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac10cd90 con 0x7fc7ac101190 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b24e3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 0x7fc7ac19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b24e3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 0x7fc7ac19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43494/0 (socket says 172.21.15.102:43494) 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b24e3700 1 -- 172.21.15.102:0/354019264 learned_addr learned my addr 172.21.15.102:0/354019264 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b1ce2700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101190 0x7fc7ac198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b14e1700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7ac102390 0x7fc7ac198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b24e3700 1 -- 172.21.15.102:0/354019264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101190 msgr2=0x7fc7ac198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b24e3700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101190 0x7fc7ac198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b24e3700 1 -- 172.21.15.102:0/354019264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7ac102390 msgr2=0x7fc7ac198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b24e3700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7ac102390 0x7fc7ac198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.883+0000 7fc7b24e3700 1 -- 172.21.15.102:0/354019264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac19fe90 con 0x7fc7ac102d30 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.884+0000 7fc7b24e3700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 0x7fc7ac19f7f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4007a90 tx=0x7fc7a4007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.884+0000 7fc7a2ffd700 1 -- 172.21.15.102:0/354019264 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4021070 con 0x7fc7ac102d30 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.884+0000 7fc7a2ffd700 1 -- 172.21.15.102:0/354019264 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7a401db00 con 0x7fc7ac102d30 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.884+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac1a0120 con 0x7fc7ac102d30 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.884+0000 7fc7a2ffd700 1 -- 172.21.15.102:0/354019264 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a401bb60 con 0x7fc7ac102d30 2024-04-24T23:06:16.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.884+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac1a05d0 con 0x7fc7ac102d30 2024-04-24T23:06:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.885+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ac068f20 con 0x7fc7ac102d30 2024-04-24T23:06:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.886+0000 7fc7a2ffd700 1 -- 172.21.15.102:0/354019264 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fc7a4024070 con 0x7fc7ac102d30 2024-04-24T23:06:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.886+0000 7fc7a2ffd700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7980a2c10 0x7fc7980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.887+0000 7fc7b1ce2700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7980a2c10 0x7fc7980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.887+0000 7fc7a2ffd700 1 -- 172.21.15.102:0/354019264 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7fc7a40cf660 con 0x7fc7ac102d30 2024-04-24T23:06:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.887+0000 7fc7b1ce2700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7980a2c10 0x7fc7980a50d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac1021f0 tx=0x7fc7a800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:16.890+0000 7fc7a2ffd700 1 -- 172.21.15.102:0/354019264 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc7a409db10 con 0x7fc7ac102d30 2024-04-24T23:06:17.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:17.095+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc7ac1a0830 con 0x7fc7980a2c10 2024-04-24T23:06:17.715 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:17 smithi102 ceph-mon[29216]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:17.715 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:06:17.715 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:06:17.715 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:17 smithi102 ceph-mon[29216]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:06:17.715 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:06:17.715 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:17 smithi102 ceph-mon[29216]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:17.715 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:17.715 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:17.715 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:17 smithi183 ceph-mon[33137]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:17 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:06:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:17 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:06:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:17 smithi183 ceph-mon[33137]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:06:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:17 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:06:17.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:17 smithi183 ceph-mon[33137]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:17.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:17 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:17.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:17 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:17.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:17 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:17 smithi161 ceph-mon[32709]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:17 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:06:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:17 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:06:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:17 smithi161 ceph-mon[32709]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:06:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:17 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:06:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:17 smithi161 ceph-mon[32709]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:17 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:17.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:17 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:17.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:17 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:18.025 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:06:17 smithi102 sudo[49115]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:06:18.025 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:06:17 smithi102 sudo[49115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:18.025 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:06:17 smithi102 sudo[49115]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:18.026 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:06:17 smithi102 sudo[49115]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:18.322 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:06:18 smithi102 sudo[49135]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:06:18.322 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:06:18 smithi102 sudo[49135]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:18.322 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:06:18 smithi102 sudo[49135]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:18.322 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:06:18 smithi102 sudo[49135]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:18.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:18 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:06:18.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:18 smithi161 ceph-mon[32709]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:06:18.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:18 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:18.606 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:18 smithi161 ceph-mon[32709]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:18.606 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:06:18 smithi161 sudo[41408]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:06:18.606 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:06:18 smithi161 sudo[41408]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:18.606 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:06:18 smithi161 sudo[41408]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:18.607 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:06:18 smithi161 sudo[41408]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:18.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:18 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:06:18.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:18 smithi102 ceph-mon[29216]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:06:18.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:18 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:18.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:18 smithi102 ceph-mon[29216]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:18.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:18 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:06:18.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:18 smithi183 ceph-mon[33137]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:06:18.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:18 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:18.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:18 smithi183 ceph-mon[33137]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:18.859 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:06:18 smithi161 sudo[41411]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:06:18.859 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:06:18 smithi161 sudo[41411]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:18.859 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:06:18 smithi161 sudo[41411]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:18.860 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:06:18 smithi161 sudo[41411]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:19.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:19.119+0000 7fc7a2ffd700 1 -- 172.21.15.102:0/354019264 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc7a4099a80 con 0x7fc7ac102d30 2024-04-24T23:06:19.124 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:18 smithi183 sudo[42339]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:06:19.124 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:18 smithi183 sudo[42339]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:19.124 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:18 smithi183 sudo[42339]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:19.124 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:18 smithi183 sudo[42339]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:19.512 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:19 smithi183 sudo[42342]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:06:19.512 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:19 smithi183 sudo[42342]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:19.512 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:19 smithi183 sudo[42342]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:19.512 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:06:19 smithi183 sudo[42342]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:19.666 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 sudo[49175]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:06:19.666 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 sudo[49175]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:19.666 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 sudo[49175]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:19.666 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 sudo[49175]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:19.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:19 smithi183 ceph-mon[33137]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:06:19.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:19.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:19 smithi183 ceph-mon[33137]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:06:19.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:19.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:19 smithi183 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:06:19.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:19 smithi183 sudo[42345]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:06:19.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:19 smithi183 sudo[42345]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:19.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:19 smithi183 sudo[42345]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:19.924 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:19 smithi183 sudo[42345]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:20.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 sudo[49214]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:06:20.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 sudo[49214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:20.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 sudo[49214]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:20.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 sudo[49214]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:20.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 ceph-mon[29216]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:06:20.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:20.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 ceph-mon[29216]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:06:20.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:20.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:19 smithi102 ceph-mon[29216]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:06:20.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:19 smithi161 ceph-mon[32709]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:06:20.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:20.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:19 smithi161 ceph-mon[32709]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:06:20.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:20.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:19 smithi161 ceph-mon[32709]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:06:20.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 sudo[42348]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:06:20.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 sudo[42348]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:20.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 sudo[42348]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:20.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 sudo[42348]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:20.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 sudo[41414]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:06:20.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 sudo[41414]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:20.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 sudo[41414]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:20.651 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 sudo[41414]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:20.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:06:20.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:20.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:20.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:20.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:06:20.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:20.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:20.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:20.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2805060818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aea12510-9a47-49d7-b0ca-ee38b19c3249"}]: dispatch 2024-04-24T23:06:20.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:06:20.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:20.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:06:20.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:20.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:20.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:20.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2805060818' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aea12510-9a47-49d7-b0ca-ee38b19c3249"}]': finished 2024-04-24T23:06:20.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:06:20.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:20.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:20 smithi102 ceph-mon[29216]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:06:21.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:06:21.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:21.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:21.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:21.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2805060818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aea12510-9a47-49d7-b0ca-ee38b19c3249"}]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2805060818' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aea12510-9a47-49d7-b0ca-ee38b19c3249"}]': finished 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:21.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:20 smithi183 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:06:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 sudo[41417]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:06:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 sudo[41417]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:06:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 sudo[41417]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:06:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 sudo[41417]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2805060818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aea12510-9a47-49d7-b0ca-ee38b19c3249"}]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:21.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:21.112 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2805060818' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aea12510-9a47-49d7-b0ca-ee38b19c3249"}]': finished 2024-04-24T23:06:21.112 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:06:21.112 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:21.112 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:20 smithi161 ceph-mon[32709]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:06:22.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:21 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2434443783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:22.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:21 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2434443783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:22.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:21 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2434443783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:23.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:22 smithi183 ceph-mon[33137]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:23.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:22 smithi102 ceph-mon[29216]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:23.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:22 smithi161 ceph-mon[32709]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:25.473 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:25 smithi102 ceph-mon[29216]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:25.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:25 smithi183 ceph-mon[33137]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:25 smithi161 ceph-mon[32709]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:27.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:27 smithi102 ceph-mon[29216]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:27 smithi183 ceph-mon[33137]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:27 smithi161 ceph-mon[32709]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:28.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:06:28.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:28.510 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:28 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:06:28.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:28 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:28.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:28 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:06:28.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:28 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:29.484 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:29 smithi102 ceph-mon[29216]: Deploying daemon osd.3 on smithi102 2024-04-24T23:06:29.484 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:29 smithi102 ceph-mon[29216]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:29.503 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:29 smithi183 ceph-mon[33137]: Deploying daemon osd.3 on smithi102 2024-04-24T23:06:29.503 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:29 smithi183 ceph-mon[33137]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:29 smithi161 ceph-mon[32709]: Deploying daemon osd.3 on smithi102 2024-04-24T23:06:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:29 smithi161 ceph-mon[32709]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:31 smithi183 ceph-mon[33137]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:31.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:31 smithi102 ceph-mon[29216]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:31 smithi161 ceph-mon[32709]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:33.354 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:33 smithi102 ceph-mon[29216]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:33.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:33 smithi183 ceph-mon[33137]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:33.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:33 smithi161 ceph-mon[32709]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:34.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:34 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:34.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:34 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:34.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:34 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:34.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:34 smithi183 ceph-mon[33137]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:34.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:34 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:34.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:34 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:34.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:34 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:34.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:34 smithi102 ceph-mon[29216]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:34.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:34 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:34.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:34 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:34.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:34 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:34.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:34 smithi161 ceph-mon[32709]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:37.407 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 3 on host 'smithi102' 2024-04-24T23:06:37.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.405+0000 7fc7a2ffd700 1 -- 172.21.15.102:0/354019264 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc7ac1a0830 con 0x7fc7980a2c10 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7980a2c10 msgr2=0x7fc7980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7980a2c10 0x7fc7980a50d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac1021f0 tx=0x7fc7a800a3b0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 msgr2=0x7fc7ac19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 0x7fc7ac19f7f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4007a90 tx=0x7fc7a4007ac0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 shutdown_connections 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7980a2c10 0x7fc7980a50d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7ac102d30 0x7fc7ac19f7f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101190 0x7fc7ac198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 --2- 172.21.15.102:0/354019264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7ac102390 0x7fc7ac198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 >> 172.21.15.102:0/354019264 conn(0x7fc7ac0fc730 msgr2=0x7fc7ac069f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 shutdown_connections 2024-04-24T23:06:37.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:06:37.408+0000 7fc7b3f46700 1 -- 172.21.15.102:0/354019264 wait complete. 2024-04-24T23:06:37.422 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:37 smithi102 ceph-mon[29216]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:37.423 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:37 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:37.423 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:37 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:37.423 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:37 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:37.423 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:37 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:37.423 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:37 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:37 smithi183 ceph-mon[33137]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:37 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:37 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:37 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:37 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:37 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:37 smithi161 ceph-mon[32709]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:37 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:37 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:37 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:37.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:37 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:37.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:37 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:38.134 DEBUG:teuthology.orchestra.run.smithi102:osd.3> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T23:06:38.361 INFO:tasks.cephadm:Deploying osd.4 on smithi161 with /dev/vg_nvme/lv_3... 2024-04-24T23:06:38.361 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:06:38.369 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:38 smithi102 ceph-mon[29216]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:06:38.369 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:38.369 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:38.369 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:38 smithi183 ceph-mon[33137]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:06:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:38 smithi161 ceph-mon[32709]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:06:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:38 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:38 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:38 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:38.610 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:06:39.326 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:39 smithi161 ceph-mon[32709]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:39.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:39 smithi161 ceph-mon[32709]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:06:39.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:39 smithi161 ceph-mon[32709]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:06:39.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:39 smithi161 ceph-mon[32709]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:06:39.327 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:39.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:39 smithi183 ceph-mon[33137]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:39.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:39 smithi183 ceph-mon[33137]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:06:39.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:39 smithi183 ceph-mon[33137]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:06:39.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:39 smithi183 ceph-mon[33137]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:06:39.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:39.519 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:39 smithi102 ceph-mon[29216]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:39.519 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:39 smithi102 ceph-mon[29216]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:06:39.519 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:39 smithi102 ceph-mon[29216]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:06:39.519 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:39 smithi102 ceph-mon[29216]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:06:39.520 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:39.520 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:06:39 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3[51263]: 2024-04-24T23:06:39.235+0000 7f0254207700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T23:06:39.520 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:06:39 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3[51263]: 2024-04-24T23:06:39.240+0000 7f024fffa700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:06:40.266 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:40 smithi161 ceph-mon[32709]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:06:40.266 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:40 smithi161 ceph-mon[32709]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:06:40.266 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:40 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:40.266 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:40 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:40.266 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:40 smithi161 ceph-mon[32709]: osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] boot 2024-04-24T23:06:40.267 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:40 smithi161 ceph-mon[32709]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:06:40.267 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:40 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:40.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:40 smithi183 ceph-mon[33137]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:06:40.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:40 smithi183 ceph-mon[33137]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:06:40.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:40 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:40.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:40 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:40.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:40 smithi183 ceph-mon[33137]: osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] boot 2024-04-24T23:06:40.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:40 smithi183 ceph-mon[33137]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:06:40.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:40 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:40.522 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:40 smithi102 ceph-mon[29216]: from='osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:06:40.522 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:40 smithi102 ceph-mon[29216]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:06:40.522 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:40 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:40.522 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:40 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:40.522 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:40 smithi102 ceph-mon[29216]: osd.3 [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] boot 2024-04-24T23:06:40.522 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:40 smithi102 ceph-mon[29216]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:06:40.522 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:40 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:06:40.828 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:06:40.828 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:06:40.828 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-04-24T23:06:40.829 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-04-24T23:06:40.829 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113874 s, 921 MB/s 2024-04-24T23:06:40.829 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:41.478 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:41.479 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:41.505 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-04-24T23:06:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:06:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:06:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:06:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:41.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:41.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:41.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:41.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:41.763 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:06:42.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:42 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi102 2024-04-24T23:06:42.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:42 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi102 to 8544M 2024-04-24T23:06:42.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:42.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:42 smithi161 ceph-mon[32709]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:06:42.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:42 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi102 2024-04-24T23:06:42.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:42 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi102 to 8544M 2024-04-24T23:06:42.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:42.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:42 smithi183 ceph-mon[33137]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:06:42.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:42 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi102 2024-04-24T23:06:42.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:42 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi102 to 8544M 2024-04-24T23:06:42.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:42.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:42 smithi102 ceph-mon[29216]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:06:43.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.363+0000 7f0600967700 1 -- 172.21.15.161:0/1184316305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 msgr2=0x7f05fc1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.363+0000 7f0600967700 1 --2- 172.21.15.161:0/1184316305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 0x7f05fc1038f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0099c0 tx=0x7f05ec009cd0 comp rx=0 tx=0).stop 2024-04-24T23:06:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.363+0000 7f0600967700 1 -- 172.21.15.161:0/1184316305 shutdown_connections 2024-04-24T23:06:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.363+0000 7f0600967700 1 --2- 172.21.15.161:0/1184316305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05fc103e30 0x7f05fc10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.363+0000 7f0600967700 1 --2- 172.21.15.161:0/1184316305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05fc102290 0x7f05fc1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.363+0000 7f0600967700 1 --2- 172.21.15.161:0/1184316305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 0x7f05fc1038f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.363+0000 7f0600967700 1 -- 172.21.15.161:0/1184316305 >> 172.21.15.161:0/1184316305 conn(0x7f05fc0fd830 msgr2=0x7f05fc0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.363+0000 7f0600967700 1 -- 172.21.15.161:0/1184316305 shutdown_connections 2024-04-24T23:06:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.363+0000 7f0600967700 1 -- 172.21.15.161:0/1184316305 wait complete. 2024-04-24T23:06:43.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f0600967700 1 Processor -- start 2024-04-24T23:06:43.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f0600967700 1 -- start start 2024-04-24T23:06:43.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f0600967700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05fc102290 0x7f05fc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:43.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f0600967700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 0x7f05fc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:43.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f0600967700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05fc103e30 0x7f05fc19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:43.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f0600967700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc10dd80 con 0x7f05fc102290 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f0600967700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc10dc00 con 0x7f05fc103490 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f0600967700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc10df00 con 0x7f05fc103e30 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f05f9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 0x7f05fc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f05f9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 0x7f05fc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50812/0 (socket says 172.21.15.161:50812) 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f05f9d9b700 1 -- 172.21.15.161:0/1903264103 learned_addr learned my addr 172.21.15.161:0/1903264103 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f05fa59c700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05fc102290 0x7f05fc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f05fad9d700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05fc103e30 0x7f05fc19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f05f9d9b700 1 -- 172.21.15.161:0/1903264103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05fc103e30 msgr2=0x7f05fc19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.364+0000 7f05f9d9b700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05fc103e30 0x7f05fc19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.365+0000 7f05f9d9b700 1 -- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05fc102290 msgr2=0x7f05fc198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.365+0000 7f05f9d9b700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05fc102290 0x7f05fc198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.365+0000 7f05f9d9b700 1 -- 172.21.15.161:0/1903264103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc074460 con 0x7f05fc103490 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.365+0000 7f05f9d9b700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 0x7f05fc198990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05ec006010 tx=0x7f05ec0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:43.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.365+0000 7f05eb7fe700 1 -- 172.21.15.161:0/1903264103 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec027070 con 0x7f05fc103490 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.365+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc0746f0 con 0x7f05fc103490 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.365+0000 7f05eb7fe700 1 -- 172.21.15.161:0/1903264103 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05ec004050 con 0x7f05fc103490 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.365+0000 7f05eb7fe700 1 -- 172.21.15.161:0/1903264103 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec022cb0 con 0x7f05fc103490 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.365+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc074c50 con 0x7f05fc103490 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.366+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc068f20 con 0x7f05fc103490 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.367+0000 7f05eb7fe700 1 -- 172.21.15.161:0/1903264103 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f05ec02a070 con 0x7f05fc103490 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.367+0000 7f05eb7fe700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05e40a2c20 0x7f05e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.368+0000 7f05eb7fe700 1 -- 172.21.15.161:0/1903264103 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f05ec0cf280 con 0x7f05fc103490 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.368+0000 7f05fa59c700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05e40a2c20 0x7f05e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.368+0000 7f05fa59c700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05e40a2c20 0x7f05e40a50e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f05fc1032f0 tx=0x7f05f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:43.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.371+0000 7f05eb7fe700 1 -- 172.21.15.161:0/1903264103 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f05ec013070 con 0x7f05fc103490 2024-04-24T23:06:43.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:43 smithi183 ceph-mon[33137]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:43.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:43 smithi102 ceph-mon[29216]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:43.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:06:43.576+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f05fc109200 con 0x7f05e40a2c20 2024-04-24T23:06:43.578 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:43 smithi161 ceph-mon[32709]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:44.506 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:44 smithi161 ceph-mon[32709]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:06:44.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:44.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:44.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:44.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:44 smithi183 ceph-mon[33137]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:06:44.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:44.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:44.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:44.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:44 smithi102 ceph-mon[29216]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:06:44.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:44.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:44.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:45.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:45 smithi161 ceph-mon[32709]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:45.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:45 smithi161 ceph-mon[32709]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:45.433 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:45 smithi161 ceph-mon[32709]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:06:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:45 smithi183 ceph-mon[33137]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:45 smithi183 ceph-mon[33137]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:45 smithi183 ceph-mon[33137]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:06:45.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:45 smithi102 ceph-mon[29216]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:45.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:45 smithi102 ceph-mon[29216]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:45.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:45 smithi102 ceph-mon[29216]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:06:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:47 smithi183 ceph-mon[33137]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:47 smithi183 ceph-mon[33137]: from='client.? 172.21.15.161:0/1928739125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1d828f2-40e4-4c0b-9ffe-41ceb2115033"}]: dispatch 2024-04-24T23:06:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:47 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1d828f2-40e4-4c0b-9ffe-41ceb2115033"}]: dispatch 2024-04-24T23:06:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:47 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1d828f2-40e4-4c0b-9ffe-41ceb2115033"}]': finished 2024-04-24T23:06:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:47 smithi183 ceph-mon[33137]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:06:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:47 smithi183 ceph-mon[33137]: from='client.? 172.21.15.161:0/2738024787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:47 smithi102 ceph-mon[29216]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:47 smithi102 ceph-mon[29216]: from='client.? 172.21.15.161:0/1928739125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1d828f2-40e4-4c0b-9ffe-41ceb2115033"}]: dispatch 2024-04-24T23:06:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:47 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1d828f2-40e4-4c0b-9ffe-41ceb2115033"}]: dispatch 2024-04-24T23:06:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:47 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1d828f2-40e4-4c0b-9ffe-41ceb2115033"}]': finished 2024-04-24T23:06:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:47 smithi102 ceph-mon[29216]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:06:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:47 smithi102 ceph-mon[29216]: from='client.? 172.21.15.161:0/2738024787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:47 smithi161 ceph-mon[32709]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:06:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:47 smithi161 ceph-mon[32709]: from='client.? 172.21.15.161:0/1928739125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1d828f2-40e4-4c0b-9ffe-41ceb2115033"}]: dispatch 2024-04-24T23:06:47.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:47 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1d828f2-40e4-4c0b-9ffe-41ceb2115033"}]: dispatch 2024-04-24T23:06:47.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:47 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1d828f2-40e4-4c0b-9ffe-41ceb2115033"}]': finished 2024-04-24T23:06:47.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:47 smithi161 ceph-mon[32709]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:06:47.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:06:47.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:47 smithi161 ceph-mon[32709]: from='client.? 172.21.15.161:0/2738024787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:49.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:49 smithi102 ceph-mon[29216]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:06:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:49 smithi161 ceph-mon[32709]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:06:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:49 smithi183 ceph-mon[33137]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:06:51.421 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:51 smithi161 ceph-mon[32709]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:06:51.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:51 smithi102 ceph-mon[29216]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:06:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:51 smithi183 ceph-mon[33137]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:06:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:53 smithi102 ceph-mon[29216]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:06:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:53 smithi161 ceph-mon[32709]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:06:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:53 smithi183 ceph-mon[33137]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:06:54.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:06:54.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:54.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:06:54.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:54.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:06:54.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:55.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:55 smithi102 ceph-mon[29216]: Deploying daemon osd.4 on smithi161 2024-04-24T23:06:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:55 smithi102 ceph-mon[29216]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:06:55.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:55 smithi161 ceph-mon[32709]: Deploying daemon osd.4 on smithi161 2024-04-24T23:06:55.593 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:55 smithi161 ceph-mon[32709]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:06:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:55 smithi183 ceph-mon[33137]: Deploying daemon osd.4 on smithi161 2024-04-24T23:06:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:55 smithi183 ceph-mon[33137]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:06:57.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:57 smithi183 ceph-mon[33137]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:06:57.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:57 smithi102 ceph-mon[29216]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:06:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:57 smithi161 ceph-mon[32709]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:06:59.475 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:59 smithi161 ceph-mon[32709]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:06:59.476 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:59 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:59.476 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:59 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:59.476 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:06:59 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:59.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:59 smithi102 ceph-mon[29216]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:06:59.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:59 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:59.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:59 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:59.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:06:59 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:59.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:59 smithi183 ceph-mon[33137]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:06:59.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:59 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:59.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:59 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:06:59.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:06:59 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:01.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:01 smithi161 ceph-mon[32709]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:01.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:01 smithi102 ceph-mon[29216]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:01 smithi183 ceph-mon[33137]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:02.594 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 4 on host 'smithi161' 2024-04-24T23:07:02.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.592+0000 7f05eb7fe700 1 -- 172.21.15.161:0/1903264103 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f05fc109200 con 0x7f05e40a2c20 2024-04-24T23:07:02.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05e40a2c20 msgr2=0x7f05e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:02.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05e40a2c20 0x7f05e40a50e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f05fc1032f0 tx=0x7f05f000b410 comp rx=0 tx=0).stop 2024-04-24T23:07:02.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 msgr2=0x7f05fc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:02.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 0x7f05fc198990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05ec006010 tx=0x7f05ec0047b0 comp rx=0 tx=0).stop 2024-04-24T23:07:02.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 shutdown_connections 2024-04-24T23:07:02.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05fc103e30 0x7f05fc19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:02.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05e40a2c20 0x7f05e40a50e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:02.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05fc102290 0x7f05fc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:02.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 --2- 172.21.15.161:0/1903264103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05fc103490 0x7f05fc198990 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:02.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 >> 172.21.15.161:0/1903264103 conn(0x7f05fc0fd830 msgr2=0x7f05fc106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:02.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 shutdown_connections 2024-04-24T23:07:02.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:02.594+0000 7f0600967700 1 -- 172.21.15.161:0/1903264103 wait complete. 2024-04-24T23:07:02.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:02.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:02.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:02.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:02.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:02.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:02.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:02.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:02.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:02.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='osd.4 [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:07:02.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:02 smithi161 ceph-mon[32709]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:07:03.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:03.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:03.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:03.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:03.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='osd.4 [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:07:03.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:02 smithi102 ceph-mon[29216]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:07:03.254 DEBUG:teuthology.orchestra.run.smithi161:osd.4> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T23:07:03.256 INFO:tasks.cephadm:Deploying osd.5 on smithi183 with /dev/vg_nvme/lv_3... 2024-04-24T23:07:03.256 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='osd.4 [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:07:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:02 smithi183 ceph-mon[33137]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:07:03.501 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:07:04.731 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:04 smithi161 ceph-mon[32709]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:07:04.731 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:04 smithi161 ceph-mon[32709]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:07:04.731 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:04 smithi161 ceph-mon[32709]: from='osd.4 [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:07:04.731 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:04.732 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:04 smithi161 ceph-mon[32709]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:07:04.732 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:04 smithi161 ceph-mon[32709]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:04.732 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:07:04 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4[44225]: 2024-04-24T23:07:04.576+0000 7ff542414700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T23:07:04.732 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:07:04 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4[44225]: 2024-04-24T23:07:04.583+0000 7ff53e207700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:04.861 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:04 smithi183 ceph-mon[33137]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:07:04.861 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:04 smithi183 ceph-mon[33137]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:07:04.861 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:04 smithi183 ceph-mon[33137]: from='osd.4 [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:07:04.861 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:04.861 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:04 smithi183 ceph-mon[33137]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:07:04.861 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:04 smithi183 ceph-mon[33137]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:05.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:04 smithi102 ceph-mon[29216]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:07:05.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:04 smithi102 ceph-mon[29216]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:07:05.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:04 smithi102 ceph-mon[29216]: from='osd.4 [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:07:05.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:05.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:04 smithi102 ceph-mon[29216]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:07:05.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:04 smithi102 ceph-mon[29216]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:05.461 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:07:05.461 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:07:05.461 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-24T23:07:05.462 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-24T23:07:05.462 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113681 s, 922 MB/s 2024-04-24T23:07:05.462 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-24T23:07:05.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:05 smithi183 ceph-mon[33137]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:07:05.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:05 smithi183 ceph-mon[33137]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:07:05.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:05 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:05.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:05 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:05.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:05 smithi161 ceph-mon[32709]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:07:05.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:05 smithi161 ceph-mon[32709]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:07:05.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:05 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:05.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:05 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:06.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:05 smithi102 ceph-mon[29216]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:07:06.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:05 smithi102 ceph-mon[29216]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:07:06.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:05 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:06.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:05 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:06.153 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-04-24T23:07:06.442 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: osd.4 [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] boot 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi161 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:06.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi161 to 8544M 2024-04-24T23:07:06.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:06.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:06.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:06.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:06.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:07:06.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:07:06.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: osd.4 [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] boot 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi161 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi161 to 8544M 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:06.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: osd.4 [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] boot 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi161 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi161 to 8544M 2024-04-24T23:07:07.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:07.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:07.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:07.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:07.825 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:07 smithi183 ceph-mon[33137]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:07:07.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:07 smithi161 ceph-mon[32709]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:07:08.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.062+0000 7f4a4d051700 1 -- 172.21.15.183:0/18633760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a48102d40 msgr2=0x7f4a481095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:08.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.062+0000 7f4a4d051700 1 --2- 172.21.15.183:0/18633760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a48102d40 0x7f4a481095f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000b0a0 tx=0x7f4a4000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.062+0000 7f4a4d051700 1 -- 172.21.15.183:0/18633760 shutdown_connections 2024-04-24T23:07:08.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.062+0000 7f4a4d051700 1 --2- 172.21.15.183:0/18633760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a48102d40 0x7f4a481095f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.062+0000 7f4a4d051700 1 --2- 172.21.15.183:0/18633760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a481011a0 0x7f4a481015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.065 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.062+0000 7f4a4d051700 1 --2- 172.21.15.183:0/18633760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a481023a0 0x7f4a48102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.065 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.062+0000 7f4a4d051700 1 -- 172.21.15.183:0/18633760 >> 172.21.15.183:0/18633760 conn(0x7f4a480fc760 msgr2=0x7f4a480feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:08.065 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.062+0000 7f4a4d051700 1 -- 172.21.15.183:0/18633760 shutdown_connections 2024-04-24T23:07:08.065 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.062+0000 7f4a4d051700 1 -- 172.21.15.183:0/18633760 wait complete. 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.063+0000 7f4a4d051700 1 Processor -- start 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.063+0000 7f4a4d051700 1 -- start start 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.063+0000 7f4a4d051700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a481011a0 0x7f4a481972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.063+0000 7f4a4d051700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a481023a0 0x7f4a48197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.063+0000 7f4a4d051700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a48102d40 0x7f4a4819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4d051700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4810cc40 con 0x7f4a48102d40 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4d051700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4810cac0 con 0x7f4a481011a0 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4d051700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4810cdc0 con 0x7f4a481023a0 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a481023a0 0x7f4a48197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a481023a0 0x7f4a48197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:45006/0 (socket says 172.21.15.183:45006) 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4659c700 1 -- 172.21.15.183:0/3591562688 learned_addr learned my addr 172.21.15.183:0/3591562688 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:07:08.072 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4759e700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a48102d40 0x7f4a4819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a46d9d700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a481011a0 0x7f4a481972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4659c700 1 -- 172.21.15.183:0/3591562688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a481011a0 msgr2=0x7f4a481972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4659c700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a481011a0 0x7f4a481972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4659c700 1 -- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a48102d40 msgr2=0x7f4a4819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4659c700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a48102d40 0x7f4a4819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4659c700 1 -- 172.21.15.183:0/3591562688 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4819ef40 con 0x7f4a481023a0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4659c700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a481023a0 0x7f4a48197820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c950 tx=0x7f4a3c00cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a2ffff700 1 -- 172.21.15.183:0/3591562688 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c007ac0 con 0x7f4a481023a0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4819f230 con 0x7f4a481023a0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a2ffff700 1 -- 172.21.15.183:0/3591562688 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a3c004ce0 con 0x7f4a481023a0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.064+0000 7f4a2ffff700 1 -- 172.21.15.183:0/3591562688 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0055e0 con 0x7f4a481023a0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.065+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4819f770 con 0x7f4a481023a0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.066+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a48068f20 con 0x7f4a481023a0 2024-04-24T23:07:08.073 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.066+0000 7f4a2ffff700 1 -- 172.21.15.183:0/3591562688 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4a3c004750 con 0x7f4a481023a0 2024-04-24T23:07:08.074 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.067+0000 7f4a2ffff700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4a300a2c20 0x7f4a300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:08.074 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.067+0000 7f4a2ffff700 1 -- 172.21.15.183:0/3591562688 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f4a3c0cdb40 con 0x7f4a481023a0 2024-04-24T23:07:08.074 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.068+0000 7f4a46d9d700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4a300a2c20 0x7f4a300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:08.074 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.068+0000 7f4a46d9d700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4a300a2c20 0x7f4a300a50e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a38005950 tx=0x7f4a380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:08.074 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:07 smithi102 ceph-mon[29216]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:07:08.074 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.072+0000 7f4a2ffff700 1 -- 172.21.15.183:0/3591562688 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4a3c014030 con 0x7f4a481023a0 2024-04-24T23:07:08.279 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:08.278+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4a48062670 con 0x7f4a300a2c20 2024-04-24T23:07:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:08 smithi161 ceph-mon[32709]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:08 smithi161 ceph-mon[32709]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:08.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:08.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:08.887 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:08 smithi183 ceph-mon[33137]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:08.887 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:08 smithi183 ceph-mon[33137]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:08.887 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:08.887 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:08.887 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:09.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:08 smithi102 ceph-mon[29216]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:09.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:08 smithi102 ceph-mon[29216]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:09.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:09.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:09.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:11 smithi183 ceph-mon[33137]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:11 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/2333315590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f735dd7c-0d04-4448-846f-53949f04a067"}]: dispatch 2024-04-24T23:07:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:11 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f735dd7c-0d04-4448-846f-53949f04a067"}]: dispatch 2024-04-24T23:07:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:11 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f735dd7c-0d04-4448-846f-53949f04a067"}]': finished 2024-04-24T23:07:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:11 smithi183 ceph-mon[33137]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:07:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:11 smithi102 ceph-mon[29216]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:11 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/2333315590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f735dd7c-0d04-4448-846f-53949f04a067"}]: dispatch 2024-04-24T23:07:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:11 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f735dd7c-0d04-4448-846f-53949f04a067"}]: dispatch 2024-04-24T23:07:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:11 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f735dd7c-0d04-4448-846f-53949f04a067"}]': finished 2024-04-24T23:07:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:11 smithi102 ceph-mon[29216]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:07:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:11 smithi161 ceph-mon[32709]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:11 smithi161 ceph-mon[32709]: from='client.? 172.21.15.183:0/2333315590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f735dd7c-0d04-4448-846f-53949f04a067"}]: dispatch 2024-04-24T23:07:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:11 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f735dd7c-0d04-4448-846f-53949f04a067"}]: dispatch 2024-04-24T23:07:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:11 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f735dd7c-0d04-4448-846f-53949f04a067"}]': finished 2024-04-24T23:07:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:11 smithi161 ceph-mon[32709]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:07:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:12.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:12 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/2496795938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:12.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:12 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/2496795938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:12.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:12 smithi161 ceph-mon[32709]: from='client.? 172.21.15.183:0/2496795938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:13.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:13 smithi183 ceph-mon[33137]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:13.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:13 smithi102 ceph-mon[29216]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:13 smithi161 ceph-mon[32709]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:15 smithi183 ceph-mon[33137]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:15.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:15 smithi102 ceph-mon[29216]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:15 smithi161 ceph-mon[32709]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:17.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:17 smithi183 ceph-mon[33137]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:17.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:17 smithi102 ceph-mon[29216]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:17 smithi161 ceph-mon[32709]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:19 smithi183 ceph-mon[33137]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:07:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:19 smithi102 ceph-mon[29216]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:07:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:19 smithi161 ceph-mon[32709]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:07:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:20.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:20 smithi183 ceph-mon[33137]: Deploying daemon osd.5 on smithi183 2024-04-24T23:07:20.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:20 smithi102 ceph-mon[29216]: Deploying daemon osd.5 on smithi183 2024-04-24T23:07:20.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:20 smithi161 ceph-mon[32709]: Deploying daemon osd.5 on smithi183 2024-04-24T23:07:21.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:21 smithi183 ceph-mon[33137]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:21 smithi102 ceph-mon[29216]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:21.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:21 smithi161 ceph-mon[32709]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:23 smithi183 ceph-mon[33137]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:23.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:23 smithi102 ceph-mon[29216]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:23 smithi161 ceph-mon[32709]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:24 smithi161 ceph-mon[32709]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:24 smithi183 ceph-mon[33137]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:25.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:25.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:25.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:25.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:24 smithi102 ceph-mon[29216]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:27.361 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:27 smithi183 ceph-mon[33137]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:27.442 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 5 on host 'smithi183' 2024-04-24T23:07:27.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.440+0000 7f4a2ffff700 1 -- 172.21.15.183:0/3591562688 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a48062670 con 0x7f4a300a2c20 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4a300a2c20 msgr2=0x7f4a300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4a300a2c20 0x7f4a300a50e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a38005950 tx=0x7f4a380058e0 comp rx=0 tx=0).stop 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a481023a0 msgr2=0x7f4a48197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a481023a0 0x7f4a48197820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c950 tx=0x7f4a3c00cc60 comp rx=0 tx=0).stop 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 shutdown_connections 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4a300a2c20 0x7f4a300a50e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a48102d40 0x7f4a4819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a481011a0 0x7f4a481972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 --2- 172.21.15.183:0/3591562688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a481023a0 0x7f4a48197820 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 >> 172.21.15.183:0/3591562688 conn(0x7f4a480fc760 msgr2=0x7f4a48105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 shutdown_connections 2024-04-24T23:07:27.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:07:27.443+0000 7f4a4d051700 1 -- 172.21.15.183:0/3591562688 wait complete. 2024-04-24T23:07:27.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:27 smithi102 ceph-mon[29216]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:27 smithi161 ceph-mon[32709]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:28.133 DEBUG:teuthology.orchestra.run.smithi183:osd.5> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T23:07:28.135 INFO:tasks.cephadm:Deploying osd.6 on smithi102 with /dev/vg_nvme/lv_2... 2024-04-24T23:07:28.135 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:07:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:07:28.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:28 smithi102 ceph-mon[29216]: from='osd.5 [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:07:28.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:28 smithi102 ceph-mon[29216]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:07:28.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:28 smithi102 ceph-mon[29216]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:28.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:28 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:28 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:28 smithi183 ceph-mon[33137]: from='osd.5 [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:07:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:28 smithi183 ceph-mon[33137]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:07:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:28 smithi183 ceph-mon[33137]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:28 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:28 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.792 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:28 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.792 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:28 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.792 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:28 smithi161 ceph-mon[32709]: from='osd.5 [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:07:28.792 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:28 smithi161 ceph-mon[32709]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:07:28.792 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:28 smithi161 ceph-mon[32709]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:28.792 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:28 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:28.792 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:28 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:29 smithi183 ceph-mon[33137]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:07:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:29 smithi183 ceph-mon[33137]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:07:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:29 smithi183 ceph-mon[33137]: from='osd.5 [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:07:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:29 smithi183 ceph-mon[33137]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:07:29.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:29 smithi161 ceph-mon[32709]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:07:29.726 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:29 smithi161 ceph-mon[32709]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:07:29.727 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:29.727 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:29 smithi161 ceph-mon[32709]: from='osd.5 [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:07:29.727 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:29 smithi161 ceph-mon[32709]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:07:29.776 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:29 smithi102 ceph-mon[29216]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:07:29.776 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:29 smithi102 ceph-mon[29216]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:07:29.776 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:29.776 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:29 smithi102 ceph-mon[29216]: from='osd.5 [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:07:29.776 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:29 smithi102 ceph-mon[29216]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:07:30.189 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:07:29 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5[45242]: 2024-04-24T23:07:29.837+0000 7f12827c4700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T23:07:30.189 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:07:29 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5[45242]: 2024-04-24T23:07:29.845+0000 7f127a5af700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:30.461 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:07:30.461 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:07:30.461 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-24T23:07:30.461 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-24T23:07:30.461 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112588 s, 931 MB/s 2024-04-24T23:07:30.461 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-24T23:07:30.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:30 smithi183 ceph-mon[33137]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:07:30.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:30 smithi183 ceph-mon[33137]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:07:30.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:30 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:30.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:30 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:30.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:30 smithi183 ceph-mon[33137]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:07:30.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:30 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:30.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:30 smithi183 ceph-mon[33137]: from='osd.5 ' entity='osd.5' 2024-04-24T23:07:30.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:30 smithi183 ceph-mon[33137]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:30.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:30 smithi102 ceph-mon[29216]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:07:30.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:30 smithi102 ceph-mon[29216]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:07:30.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:30 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:30.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:30 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:30.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:30 smithi102 ceph-mon[29216]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:07:30.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:30 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:30.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:30 smithi102 ceph-mon[29216]: from='osd.5 ' entity='osd.5' 2024-04-24T23:07:30.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:30 smithi102 ceph-mon[29216]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:30.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:30 smithi161 ceph-mon[32709]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:07:30.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:30 smithi161 ceph-mon[32709]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:07:30.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:30 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:30.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:30 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:30.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:30 smithi161 ceph-mon[32709]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:07:30.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:30 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:30.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:30 smithi161 ceph-mon[32709]: from='osd.5 ' entity='osd.5' 2024-04-24T23:07:30.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:30 smithi161 ceph-mon[32709]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:07:31.191 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2024-04-24T23:07:31.439 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:07:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:31 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:07:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:31 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:07:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:31 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:31 smithi183 ceph-mon[33137]: osd.5 [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] boot 2024-04-24T23:07:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:31 smithi183 ceph-mon[33137]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:07:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:31 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:31.729 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:31 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:07:31.729 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:31 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:07:31.729 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:31 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:31.729 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:31 smithi102 ceph-mon[29216]: osd.5 [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] boot 2024-04-24T23:07:31.729 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:31 smithi102 ceph-mon[29216]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:07:31.729 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:31 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:31.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:31 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:07:31.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:31 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:07:31.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:31 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:31.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:31 smithi161 ceph-mon[32709]: osd.5 [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] boot 2024-04-24T23:07:31.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:31 smithi161 ceph-mon[32709]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:07:31.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:31 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:07:32.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi183 2024-04-24T23:07:32.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:32.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:32.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-24T23:07:32.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:32.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:32.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:07:32.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:32.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:32.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:32.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:32 smithi161 ceph-mon[32709]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:32.938 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi183 2024-04-24T23:07:32.938 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.938 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.938 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:32.938 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:32.938 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-24T23:07:32.938 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.938 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:32.939 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:32.939 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.939 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:07:32.939 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:32.939 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:32.939 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:32.939 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:32.939 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:32 smithi102 ceph-mon[29216]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi183 2024-04-24T23:07:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:33.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:32 smithi183 ceph-mon[33137]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:33.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.185+0000 7f76000f5700 1 -- 172.21.15.102:0/3343254911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 msgr2=0x7f75f8100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:33.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.185+0000 7f76000f5700 1 --2- 172.21.15.102:0/3343254911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 0x7f75f8100d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f75f400aad0 tx=0x7f75f400ade0 comp rx=0 tx=0).stop 2024-04-24T23:07:33.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.185+0000 7f76000f5700 1 -- 172.21.15.102:0/3343254911 shutdown_connections 2024-04-24T23:07:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.185+0000 7f76000f5700 1 --2- 172.21.15.102:0/3343254911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75f8102500 0x7f75f8108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.185+0000 7f76000f5700 1 --2- 172.21.15.102:0/3343254911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 0x7f75f8100d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.185+0000 7f76000f5700 1 --2- 172.21.15.102:0/3343254911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f8101b60 0x7f75f8101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.185+0000 7f76000f5700 1 -- 172.21.15.102:0/3343254911 >> 172.21.15.102:0/3343254911 conn(0x7f75f80fbf00 msgr2=0x7f75f80fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.185+0000 7f76000f5700 1 -- 172.21.15.102:0/3343254911 shutdown_connections 2024-04-24T23:07:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.185+0000 7f76000f5700 1 -- 172.21.15.102:0/3343254911 wait complete. 2024-04-24T23:07:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f76000f5700 1 Processor -- start 2024-04-24T23:07:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f76000f5700 1 -- start start 2024-04-24T23:07:33.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f76000f5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 0x7f75f8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f76000f5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f8101b60 0x7f75f8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f76000f5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75f8102500 0x7f75f819fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f76000f5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810bbf0 con 0x7f75f8100960 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f76000f5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810ba70 con 0x7f75f8101b60 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f76000f5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810bd70 con 0x7f75f8102500 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f75fde91700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 0x7f75f8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f75fde91700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 0x7f75f8198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44438/0 (socket says 172.21.15.102:44438) 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.186+0000 7f75fde91700 1 -- 172.21.15.102:0/4020523846 learned_addr learned my addr 172.21.15.102:0/4020523846 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75fe692700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75f8102500 0x7f75f819fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75fd690700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f8101b60 0x7f75f8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:33.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75fde91700 1 -- 172.21.15.102:0/4020523846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75f8102500 msgr2=0x7f75f819fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75fde91700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75f8102500 0x7f75f819fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75fde91700 1 -- 172.21.15.102:0/4020523846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f8101b60 msgr2=0x7f75f8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75fde91700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f8101b60 0x7f75f8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75fde91700 1 -- 172.21.15.102:0/4020523846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f81a00b0 con 0x7f75f8100960 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75fde91700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 0x7f75f8198450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75f4003c50 tx=0x7f75f4003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75eeffd700 1 -- 172.21.15.102:0/4020523846 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f40040b0 con 0x7f75f8100960 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f81a0340 con 0x7f75f8100960 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75eeffd700 1 -- 172.21.15.102:0/4020523846 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75f4004250 con 0x7f75f8100960 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f75eeffd700 1 -- 172.21.15.102:0/4020523846 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f40056e0 con 0x7f75f8100960 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.187+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f81a0870 con 0x7f75f8100960 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.188+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f8068f20 con 0x7f75f8100960 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.189+0000 7f75eeffd700 1 -- 172.21.15.102:0/4020523846 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f75f401e070 con 0x7f75f8100960 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.190+0000 7f75eeffd700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75e40a2c20 0x7f75e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.190+0000 7f75fd690700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75e40a2c20 0x7f75e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.190+0000 7f75eeffd700 1 -- 172.21.15.102:0/4020523846 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f75f40cfc50 con 0x7f75f8100960 2024-04-24T23:07:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.190+0000 7f75fd690700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75e40a2c20 0x7f75e40a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75e8006fd0 tx=0x7f75e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.195+0000 7f75eeffd700 1 -- 172.21.15.102:0/4020523846 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f75f409dae0 con 0x7f75f8100960 2024-04-24T23:07:33.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:33.400+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f75f8108790 con 0x7f75e40a2c20 2024-04-24T23:07:34.009 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:33 smithi102 ceph-mon[29216]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:07:34.010 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:34.010 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:34.010 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:34.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:33 smithi183 ceph-mon[33137]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:07:34.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:34.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:34.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:34.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:33 smithi161 ceph-mon[32709]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:07:34.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:34.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:34.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:35.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:34 smithi183 ceph-mon[33137]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:35.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:34 smithi183 ceph-mon[33137]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:07:35.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:34 smithi183 ceph-mon[33137]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:35.033 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:34 smithi102 ceph-mon[29216]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:35.034 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:34 smithi102 ceph-mon[29216]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:07:35.034 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:34 smithi102 ceph-mon[29216]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:35.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:34 smithi161 ceph-mon[32709]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:35.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:34 smithi161 ceph-mon[32709]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:07:35.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:34 smithi161 ceph-mon[32709]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:36.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:35 smithi183 ceph-mon[33137]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:07:36.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:35 smithi102 ceph-mon[29216]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:07:36.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:35 smithi161 ceph-mon[32709]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:07:37.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:36 smithi183 ceph-mon[33137]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:37.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:36 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/702121739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74c01493-6457-48d8-807a-0c6108e3d384"}]: dispatch 2024-04-24T23:07:37.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:36 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/702121739' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74c01493-6457-48d8-807a-0c6108e3d384"}]': finished 2024-04-24T23:07:37.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:36 smithi183 ceph-mon[33137]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:07:37.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:36 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:36 smithi102 ceph-mon[29216]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:36 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/702121739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74c01493-6457-48d8-807a-0c6108e3d384"}]: dispatch 2024-04-24T23:07:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:36 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/702121739' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74c01493-6457-48d8-807a-0c6108e3d384"}]': finished 2024-04-24T23:07:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:36 smithi102 ceph-mon[29216]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:07:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:36 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:37.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:36 smithi161 ceph-mon[32709]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:37.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:36 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/702121739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74c01493-6457-48d8-807a-0c6108e3d384"}]: dispatch 2024-04-24T23:07:37.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:36 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/702121739' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74c01493-6457-48d8-807a-0c6108e3d384"}]': finished 2024-04-24T23:07:37.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:36 smithi161 ceph-mon[32709]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:07:37.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:36 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:38.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:37 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1611399587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:38.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:37 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1611399587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:38.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:37 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1611399587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:39.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:38 smithi183 ceph-mon[33137]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-24T23:07:39.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:38 smithi102 ceph-mon[29216]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-24T23:07:39.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:38 smithi161 ceph-mon[32709]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-24T23:07:41.498 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:41 smithi102 ceph-mon[29216]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-24T23:07:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:41 smithi183 ceph-mon[33137]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-24T23:07:41.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:41 smithi161 ceph-mon[32709]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-24T23:07:43.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:43 smithi183 ceph-mon[33137]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:07:43.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:43 smithi102 ceph-mon[29216]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:07:43.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:43 smithi161 ceph-mon[32709]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:07:44.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:07:44.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:44.550 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:07:44.550 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:44.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:07:44.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:45 smithi183 ceph-mon[33137]: Deploying daemon osd.6 on smithi102 2024-04-24T23:07:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:45 smithi183 ceph-mon[33137]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-24T23:07:45.566 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:45 smithi102 ceph-mon[29216]: Deploying daemon osd.6 on smithi102 2024-04-24T23:07:45.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:45 smithi102 ceph-mon[29216]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-24T23:07:45.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:45 smithi161 ceph-mon[32709]: Deploying daemon osd.6 on smithi102 2024-04-24T23:07:45.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:45 smithi161 ceph-mon[32709]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-24T23:07:47.494 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:47 smithi102 ceph-mon[29216]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:07:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:47 smithi183 ceph-mon[33137]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:07:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:47 smithi161 ceph-mon[32709]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:07:49.388 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:49 smithi102 ceph-mon[29216]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:07:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:49 smithi183 ceph-mon[33137]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:07:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:49 smithi161 ceph-mon[32709]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:07:51.074 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:51.074 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:51.074 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:51.074 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:50 smithi102 ceph-mon[29216]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:50 smithi161 ceph-mon[32709]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:51.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:51.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:51.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:51.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:50 smithi183 ceph-mon[33137]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:53.471 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:53 smithi102 ceph-mon[29216]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:53.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:53 smithi183 ceph-mon[33137]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.558+0000 7f75eeffd700 1 -- 172.21.15.102:0/4020523846 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f75f8108790 con 0x7f75e40a2c20 2024-04-24T23:07:53.561 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 6 on host 'smithi102' 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75e40a2c20 msgr2=0x7f75e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75e40a2c20 0x7f75e40a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75e8006fd0 tx=0x7f75e8008040 comp rx=0 tx=0).stop 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 msgr2=0x7f75f8198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 0x7f75f8198450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75f4003c50 tx=0x7f75f4003a60 comp rx=0 tx=0).stop 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 shutdown_connections 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75f8102500 0x7f75f819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75e40a2c20 0x7f75e40a50e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75f8100960 0x7f75f8198450 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 --2- 172.21.15.102:0/4020523846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f8101b60 0x7f75f8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 >> 172.21.15.102:0/4020523846 conn(0x7f75f80fbf00 msgr2=0x7f75f8107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 shutdown_connections 2024-04-24T23:07:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:07:53.561+0000 7f76000f5700 1 -- 172.21.15.102:0/4020523846 wait complete. 2024-04-24T23:07:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:53 smithi161 ceph-mon[32709]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:54.378 DEBUG:teuthology.orchestra.run.smithi102:osd.6> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T23:07:54.380 INFO:tasks.cephadm:Deploying osd.7 on smithi161 with /dev/vg_nvme/lv_2... 2024-04-24T23:07:54.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:07:54.628 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:07:54.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:54 smithi161 ceph-mon[32709]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:07:54.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:54 smithi161 ceph-mon[32709]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:54.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.750 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:54 smithi183 ceph-mon[33137]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:07:54.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:54 smithi183 ceph-mon[33137]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:54.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:54 smithi102 ceph-mon[29216]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:07:54.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:54 smithi102 ceph-mon[29216]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:54.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:54.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:55.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:55 smithi102 ceph-mon[29216]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:07:55.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:55 smithi102 ceph-mon[29216]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:07:55.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:55 smithi102 ceph-mon[29216]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:07:55.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:55 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:55.822 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:07:55 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6[57809]: 2024-04-24T23:07:55.524+0000 7fa7395a7700 -1 osd.6 0 waiting for initial osdmap 2024-04-24T23:07:55.823 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:07:55 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6[57809]: 2024-04-24T23:07:55.532+0000 7fa73539a700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:55.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:55 smithi161 ceph-mon[32709]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:07:55.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:55 smithi161 ceph-mon[32709]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:07:55.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:55 smithi161 ceph-mon[32709]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:07:55.847 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:55 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:56.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:55 smithi183 ceph-mon[33137]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:07:56.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:55 smithi183 ceph-mon[33137]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:07:56.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:55 smithi183 ceph-mon[33137]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:07:56.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:55 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:56.589 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:56 smithi102 ceph-mon[29216]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:07:56.589 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:56 smithi102 ceph-mon[29216]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T23:07:56.589 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:56.589 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:56.589 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:56 smithi102 ceph-mon[29216]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:56.706 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:07:56.706 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:07:56.706 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-04-24T23:07:56.706 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-04-24T23:07:56.706 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0200063 s, 524 MB/s 2024-04-24T23:07:56.707 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-04-24T23:07:56.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:56 smithi161 ceph-mon[32709]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:07:56.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:56 smithi161 ceph-mon[32709]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T23:07:56.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:56.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:56.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:56 smithi161 ceph-mon[32709]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:57.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:56 smithi183 ceph-mon[33137]: from='osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:07:57.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:56 smithi183 ceph-mon[33137]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T23:07:57.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:57.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:57.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:56 smithi183 ceph-mon[33137]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:07:57.373 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-04-24T23:07:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:57 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:07:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:57 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:07:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:57 smithi161 ceph-mon[32709]: osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] boot 2024-04-24T23:07:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:57 smithi161 ceph-mon[32709]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:07:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:57.624 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:07:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:57 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:07:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:57 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:07:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:57 smithi102 ceph-mon[29216]: osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] boot 2024-04-24T23:07:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:57 smithi102 ceph-mon[29216]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:07:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:57.913 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:57 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:07:57.913 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:57 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:07:57.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:57 smithi183 ceph-mon[33137]: osd.6 [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] boot 2024-04-24T23:07:57.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:57 smithi183 ceph-mon[33137]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:07:57.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi102 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi102 to 5696M 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:58.619 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:58.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:58.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:58.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:58.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:58.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:58.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:58.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:07:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi102 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi102 to 5696M 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:58.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:58.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:58.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:58.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:58.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:58.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:58.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:07:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T23:07:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi102 2024-04-24T23:07:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi102 to 5696M 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:07:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.250+0000 7fa4a0cef700 1 -- 172.21.15.161:0/401368820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c102290 msgr2=0x7fa49c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.250+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/401368820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c102290 0x7fa49c1026b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0099c0 tx=0x7fa48c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.250+0000 7fa4a0cef700 1 -- 172.21.15.161:0/401368820 shutdown_connections 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.251+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/401368820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa49c103e30 0x7fa49c10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.251+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/401368820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c102290 0x7fa49c1026b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.251+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/401368820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa49c103490 0x7fa49c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.251+0000 7fa4a0cef700 1 -- 172.21.15.161:0/401368820 >> 172.21.15.161:0/401368820 conn(0x7fa49c0fd830 msgr2=0x7fa49c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.251+0000 7fa4a0cef700 1 -- 172.21.15.161:0/401368820 shutdown_connections 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.251+0000 7fa4a0cef700 1 -- 172.21.15.161:0/401368820 wait complete. 2024-04-24T23:07:59.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.251+0000 7fa4a0cef700 1 Processor -- start 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.251+0000 7fa4a0cef700 1 -- start start 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa4a0cef700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa49c102290 0x7fa49c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa4a0cef700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa49c103490 0x7fa49c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa4a0cef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c103e30 0x7fa49c19f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa4a0cef700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c10dd00 con 0x7fa49c102290 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa4a0cef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c10db80 con 0x7fa49c103e30 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa4a0cef700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c10de80 con 0x7fa49c103490 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c103e30 0x7fa49c19f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c103e30 0x7fa49c19f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51480/0 (socket says 172.21.15.161:51480) 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49ad9d700 1 -- 172.21.15.161:0/2145523160 learned_addr learned my addr 172.21.15.161:0/2145523160 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49a59c700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa49c102290 0x7fa49c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa499d9b700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa49c103490 0x7fa49c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49ad9d700 1 -- 172.21.15.161:0/2145523160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa49c103490 msgr2=0x7fa49c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49ad9d700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa49c103490 0x7fa49c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49ad9d700 1 -- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa49c102290 msgr2=0x7fa49c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49ad9d700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa49c102290 0x7fa49c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49ad9d700 1 -- 172.21.15.161:0/2145523160 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c1a0080 con 0x7fa49c103e30 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa49ad9d700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c103e30 0x7fa49c19f9e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa494007ee0 tx=0x7fa49400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa48b7fe700 1 -- 172.21.15.161:0/2145523160 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494004bb0 con 0x7fa49c103e30 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49c1a0370 con 0x7fa49c103e30 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa48b7fe700 1 -- 172.21.15.161:0/2145523160 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa494004d50 con 0x7fa49c103e30 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa48b7fe700 1 -- 172.21.15.161:0/2145523160 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494005910 con 0x7fa49c103e30 2024-04-24T23:07:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.252+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49c1a08b0 con 0x7fa49c103e30 2024-04-24T23:07:59.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.253+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa49c068f20 con 0x7fa49c103e30 2024-04-24T23:07:59.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.255+0000 7fa48b7fe700 1 -- 172.21.15.161:0/2145523160 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa49401a020 con 0x7fa49c103e30 2024-04-24T23:07:59.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.255+0000 7fa48b7fe700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4840a2c20 0x7fa4840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:59.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.255+0000 7fa48b7fe700 1 -- 172.21.15.161:0/2145523160 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fa4940cda00 con 0x7fa49c103e30 2024-04-24T23:07:59.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.255+0000 7fa49a59c700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4840a2c20 0x7fa4840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:59.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.256+0000 7fa49a59c700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4840a2c20 0x7fa4840a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa48c000c00 tx=0x7fa48c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.260+0000 7fa48b7fe700 1 -- 172.21.15.161:0/2145523160 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa494010070 con 0x7fa49c103e30 2024-04-24T23:07:59.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:07:59.461+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa49c062670 con 0x7fa4840a2c20 2024-04-24T23:08:00.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:00.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:00.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:00.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:00.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:00.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:00.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:00.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:01.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:01 smithi161 ceph-mon[32709]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:01.146 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:01 smithi161 ceph-mon[32709]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:01 smithi183 ceph-mon[33137]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:01 smithi183 ceph-mon[33137]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:01 smithi102 ceph-mon[29216]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:01 smithi102 ceph-mon[29216]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:03 smithi183 ceph-mon[33137]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:03 smithi183 ceph-mon[33137]: from='client.? 172.21.15.161:0/3192765146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761ed995-553c-4b96-9a90-aec367743007"}]: dispatch 2024-04-24T23:08:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:03 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761ed995-553c-4b96-9a90-aec367743007"}]: dispatch 2024-04-24T23:08:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:03 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "761ed995-553c-4b96-9a90-aec367743007"}]': finished 2024-04-24T23:08:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:03 smithi183 ceph-mon[33137]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:08:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:03 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:03 smithi183 ceph-mon[33137]: from='client.? 172.21.15.161:0/3864409424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:03 smithi102 ceph-mon[29216]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:03 smithi102 ceph-mon[29216]: from='client.? 172.21.15.161:0/3192765146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761ed995-553c-4b96-9a90-aec367743007"}]: dispatch 2024-04-24T23:08:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:03 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761ed995-553c-4b96-9a90-aec367743007"}]: dispatch 2024-04-24T23:08:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:03 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "761ed995-553c-4b96-9a90-aec367743007"}]': finished 2024-04-24T23:08:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:03 smithi102 ceph-mon[29216]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:08:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:03 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:03 smithi102 ceph-mon[29216]: from='client.? 172.21.15.161:0/3864409424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:03 smithi161 ceph-mon[32709]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:03 smithi161 ceph-mon[32709]: from='client.? 172.21.15.161:0/3192765146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761ed995-553c-4b96-9a90-aec367743007"}]: dispatch 2024-04-24T23:08:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:03 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761ed995-553c-4b96-9a90-aec367743007"}]: dispatch 2024-04-24T23:08:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:03 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "761ed995-553c-4b96-9a90-aec367743007"}]': finished 2024-04-24T23:08:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:03 smithi161 ceph-mon[32709]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:08:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:03 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:03 smithi161 ceph-mon[32709]: from='client.? 172.21.15.161:0/3864409424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:05 smithi183 ceph-mon[33137]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:05 smithi102 ceph-mon[29216]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:05 smithi161 ceph-mon[32709]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:07 smithi183 ceph-mon[33137]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:07.567 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:07 smithi161 ceph-mon[32709]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:07.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:07 smithi102 ceph-mon[29216]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:09 smithi183 ceph-mon[33137]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:09 smithi102 ceph-mon[29216]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:09 smithi161 ceph-mon[32709]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:10.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:10 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:08:10.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:10 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:10.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:10 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:08:10.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:10 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:10.580 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:10 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:08:10.581 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:10 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:11 smithi183 ceph-mon[33137]: Deploying daemon osd.7 on smithi161 2024-04-24T23:08:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:11 smithi183 ceph-mon[33137]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:11.537 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:11 smithi161 ceph-mon[32709]: Deploying daemon osd.7 on smithi161 2024-04-24T23:08:11.537 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:11 smithi161 ceph-mon[32709]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:11 smithi102 ceph-mon[29216]: Deploying daemon osd.7 on smithi161 2024-04-24T23:08:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:11 smithi102 ceph-mon[29216]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:13.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:13 smithi161 ceph-mon[32709]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:13.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:13 smithi183 ceph-mon[33137]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:13.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:13 smithi102 ceph-mon[29216]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:15 smithi161 ceph-mon[32709]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:15 smithi183 ceph-mon[33137]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:15.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:15 smithi102 ceph-mon[29216]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:16.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:16.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:16.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:16.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:16.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:16.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:16.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:16.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:16.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:17.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:17 smithi102 ceph-mon[29216]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:17 smithi161 ceph-mon[32709]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:17 smithi183 ceph-mon[33137]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:19.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.078+0000 7fa48b7fe700 1 -- 172.21.15.161:0/2145523160 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa49c062670 con 0x7fa4840a2c20 2024-04-24T23:08:19.079 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 7 on host 'smithi161' 2024-04-24T23:08:19.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4840a2c20 msgr2=0x7fa4840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:19.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4840a2c20 0x7fa4840a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa48c000c00 tx=0x7fa48c005fb0 comp rx=0 tx=0).stop 2024-04-24T23:08:19.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c103e30 msgr2=0x7fa49c19f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:19.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c103e30 0x7fa49c19f9e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa494007ee0 tx=0x7fa49400d3b0 comp rx=0 tx=0).stop 2024-04-24T23:08:19.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 shutdown_connections 2024-04-24T23:08:19.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c103e30 0x7fa49c19f9e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:19.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4840a2c20 0x7fa4840a50e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:19.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa49c102290 0x7fa49c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:19.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 --2- 172.21.15.161:0/2145523160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa49c103490 0x7fa49c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:19.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 >> 172.21.15.161:0/2145523160 conn(0x7fa49c0fd830 msgr2=0x7fa49c106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:19.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 shutdown_connections 2024-04-24T23:08:19.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:08:19.080+0000 7fa4a0cef700 1 -- 172.21.15.161:0/2145523160 wait complete. 2024-04-24T23:08:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:19 smithi161 ceph-mon[32709]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:19 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:19 smithi183 ceph-mon[33137]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:19 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:19.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:19 smithi102 ceph-mon[29216]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:19 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:19.794 DEBUG:teuthology.orchestra.run.smithi161:osd.7> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T23:08:19.796 INFO:tasks.cephadm:Deploying osd.8 on smithi183 with /dev/vg_nvme/lv_2... 2024-04-24T23:08:19.796 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:08:20.044 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:08:20.313 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:20 smithi161 ceph-mon[32709]: from='osd.7 [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:08:20.313 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:20 smithi161 ceph-mon[32709]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:08:20.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:20 smithi183 ceph-mon[33137]: from='osd.7 [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:08:20.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:20 smithi183 ceph-mon[33137]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:08:20.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:20 smithi102 ceph-mon[29216]: from='osd.7 [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:08:20.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:20 smithi102 ceph-mon[29216]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:08:21.282 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:21 smithi161 ceph-mon[32709]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:21.282 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:21 smithi161 ceph-mon[32709]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:08:21.282 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:21 smithi161 ceph-mon[32709]: from='osd.7 [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:08:21.283 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:21 smithi161 ceph-mon[32709]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:08:21.283 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:21 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:21.283 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:21 smithi161 ceph-mon[32709]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:08:21.283 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:08:21 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7[50762]: 2024-04-24T23:08:21.196+0000 7f0b50c8e700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T23:08:21.283 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:08:21 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7[50762]: 2024-04-24T23:08:21.206+0000 7f0b4ca81700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:21.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:21 smithi183 ceph-mon[33137]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:21.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:21 smithi183 ceph-mon[33137]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:08:21.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:21 smithi183 ceph-mon[33137]: from='osd.7 [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:08:21.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:21 smithi183 ceph-mon[33137]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:08:21.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:21 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:21.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:21 smithi183 ceph-mon[33137]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:08:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:21 smithi102 ceph-mon[29216]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:21 smithi102 ceph-mon[29216]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:08:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:21 smithi102 ceph-mon[29216]: from='osd.7 [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:08:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:21 smithi102 ceph-mon[29216]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:08:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:21 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:21 smithi102 ceph-mon[29216]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:08:22.209 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:08:22.209 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:08:22.209 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-24T23:08:22.209 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-24T23:08:22.209 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114628 s, 915 MB/s 2024-04-24T23:08:22.209 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-24T23:08:22.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:22 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:08:22.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:22 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:08:22.507 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:22 smithi161 ceph-mon[32709]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:08:22.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:22 smithi161 ceph-mon[32709]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T23:08:22.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:22 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:22.508 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:22 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:22.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:22 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:08:22.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:22 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:08:22.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:22 smithi183 ceph-mon[33137]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:08:22.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:22 smithi183 ceph-mon[33137]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T23:08:22.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:22 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:22.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:22 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:22.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:22 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:08:22.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:22 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:08:22.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:22 smithi102 ceph-mon[29216]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:08:22.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:22 smithi102 ceph-mon[29216]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T23:08:22.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:22 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:22.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:22 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:22.961 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-04-24T23:08:23.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:23.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:23.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: osd.7 [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] boot 2024-04-24T23:08:23.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:08:23.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:23.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.211 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:08:23.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:23.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:23.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:23.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:23.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:23.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:23 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: osd.7 [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] boot 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:23.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:23.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:23 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:08:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: osd.7 [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] boot 2024-04-24T23:08:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:23.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:23 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:24.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:24 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi161 2024-04-24T23:08:24.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:24 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi161 to 5696M 2024-04-24T23:08:24.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:24 smithi183 ceph-mon[33137]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:08:24.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:24 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi161 2024-04-24T23:08:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:24 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi161 to 5696M 2024-04-24T23:08:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:24 smithi102 ceph-mon[29216]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:08:24.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:24 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi161 2024-04-24T23:08:24.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:24 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi161 to 5696M 2024-04-24T23:08:24.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:24 smithi161 ceph-mon[32709]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:08:24.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.783+0000 7f53ada7e700 1 -- 172.21.15.183:0/3385294403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 msgr2=0x7f53a810a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:24.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.783+0000 7f53ada7e700 1 --2- 172.21.15.183:0/3385294403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 0x7f53a810a770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f53a000b0a0 tx=0x7f53a000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:08:24.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.783+0000 7f53ada7e700 1 -- 172.21.15.183:0/3385294403 shutdown_connections 2024-04-24T23:08:24.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.783+0000 7f53ada7e700 1 --2- 172.21.15.183:0/3385294403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 0x7f53a810a770 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:24.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.783+0000 7f53ada7e700 1 --2- 172.21.15.183:0/3385294403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a8100540 0x7f53a8100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:24.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.783+0000 7f53ada7e700 1 --2- 172.21.15.183:0/3385294403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53a8103520 0x7f53a8103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:24.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.783+0000 7f53ada7e700 1 -- 172.21.15.183:0/3385294403 >> 172.21.15.183:0/3385294403 conn(0x7f53a80fbec0 msgr2=0x7f53a80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:24.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.783+0000 7f53ada7e700 1 -- 172.21.15.183:0/3385294403 shutdown_connections 2024-04-24T23:08:24.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.783+0000 7f53ada7e700 1 -- 172.21.15.183:0/3385294403 wait complete. 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.784+0000 7f53ada7e700 1 Processor -- start 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.784+0000 7f53ada7e700 1 -- start start 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.784+0000 7f53ada7e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53a8100540 0x7f53a8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53ada7e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a8103520 0x7f53a8198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53ada7e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 0x7f53a819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53ada7e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810dce0 con 0x7f53a8103520 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53ada7e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810db60 con 0x7f53a8100540 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53ada7e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810de60 con 0x7f53a8103ec0 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 0x7f53a819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 0x7f53a819f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38728/0 (socket says 172.21.15.183:38728) 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a7fff700 1 -- 172.21.15.183:0/1224550422 learned_addr learned my addr 172.21.15.183:0/1224550422 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a77fe700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53a8100540 0x7f53a8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a6ffd700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a8103520 0x7f53a8198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a7fff700 1 -- 172.21.15.183:0/1224550422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53a8100540 msgr2=0x7f53a8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:24.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a7fff700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53a8100540 0x7f53a8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a7fff700 1 -- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a8103520 msgr2=0x7f53a8198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a7fff700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a8103520 0x7f53a8198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a7fff700 1 -- 172.21.15.183:0/1224550422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a819ff60 con 0x7f53a8103ec0 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a6ffd700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a8103520 0x7f53a8198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a77fe700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53a8100540 0x7f53a8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a7fff700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 0x7f53a819f8c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009850 tx=0x7f53a0004d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a4ff9700 1 -- 172.21.15.183:0/1224550422 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0021070 con 0x7f53a8103ec0 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a4ff9700 1 -- 172.21.15.183:0/1224550422 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53a0007c00 con 0x7f53a8103ec0 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a81a01f0 con 0x7f53a8103ec0 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53a4ff9700 1 -- 172.21.15.183:0/1224550422 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a00052a0 con 0x7f53a8103ec0 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.785+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a81a0780 con 0x7f53a8103ec0 2024-04-24T23:08:24.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.786+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a8068f20 con 0x7f53a8103ec0 2024-04-24T23:08:24.791 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.788+0000 7f53a4ff9700 1 -- 172.21.15.183:0/1224550422 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f53a0024070 con 0x7f53a8103ec0 2024-04-24T23:08:24.791 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.788+0000 7f53a4ff9700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f53900a2ca0 0x7f53900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:24.791 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.788+0000 7f53a4ff9700 1 -- 172.21.15.183:0/1224550422 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f53a000f020 con 0x7f53a8103ec0 2024-04-24T23:08:24.792 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.788+0000 7f53a77fe700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f53900a2ca0 0x7f53900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:24.792 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.789+0000 7f53a77fe700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f53900a2ca0 0x7f53900a5160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53a8069d10 tx=0x7f5398007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:24.793 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.792+0000 7f53a4ff9700 1 -- 172.21.15.183:0/1224550422 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f53a002b020 con 0x7f53a8103ec0 2024-04-24T23:08:24.999 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:24.998+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f53a8062670 con 0x7f53900a2ca0 2024-04-24T23:08:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:25 smithi183 ceph-mon[33137]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:25 smithi183 ceph-mon[33137]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:08:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:25 smithi183 ceph-mon[33137]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:08:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:25 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:25 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:25 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:25.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:25 smithi102 ceph-mon[29216]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:25.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:25 smithi102 ceph-mon[29216]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:08:25.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:25 smithi102 ceph-mon[29216]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:08:25.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:25 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:25.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:25 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:25.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:25 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:25 smithi161 ceph-mon[32709]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:25 smithi161 ceph-mon[32709]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:08:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:25 smithi161 ceph-mon[32709]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:08:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:25 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:25 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:25.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:25 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:26.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:26 smithi183 ceph-mon[33137]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:26.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:26 smithi183 ceph-mon[33137]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T23:08:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:26 smithi102 ceph-mon[29216]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:26.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:26 smithi102 ceph-mon[29216]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T23:08:26.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:26 smithi161 ceph-mon[32709]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:26.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:26 smithi161 ceph-mon[32709]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T23:08:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:27 smithi183 ceph-mon[33137]: pgmap v185: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:27.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:27 smithi102 ceph-mon[29216]: pgmap v185: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:27 smithi161 ceph-mon[32709]: pgmap v185: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:28.510 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:28 smithi161 ceph-mon[32709]: from='client.? 172.21.15.183:0/1266251185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6396e17-1338-4c85-8e94-523088a41caf"}]: dispatch 2024-04-24T23:08:28.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:28 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6396e17-1338-4c85-8e94-523088a41caf"}]: dispatch 2024-04-24T23:08:28.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:28 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6396e17-1338-4c85-8e94-523088a41caf"}]': finished 2024-04-24T23:08:28.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:28 smithi161 ceph-mon[32709]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:08:28.511 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:28 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:28.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:28 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/1266251185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6396e17-1338-4c85-8e94-523088a41caf"}]: dispatch 2024-04-24T23:08:28.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:28 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6396e17-1338-4c85-8e94-523088a41caf"}]: dispatch 2024-04-24T23:08:28.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:28 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6396e17-1338-4c85-8e94-523088a41caf"}]': finished 2024-04-24T23:08:28.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:28 smithi183 ceph-mon[33137]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:08:28.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:28 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:28 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/1266251185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6396e17-1338-4c85-8e94-523088a41caf"}]: dispatch 2024-04-24T23:08:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:28 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6396e17-1338-4c85-8e94-523088a41caf"}]: dispatch 2024-04-24T23:08:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:28 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6396e17-1338-4c85-8e94-523088a41caf"}]': finished 2024-04-24T23:08:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:28 smithi102 ceph-mon[29216]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:08:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:29 smithi183 ceph-mon[33137]: pgmap v187: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:29 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/3894414454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:29.530 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:29 smithi161 ceph-mon[32709]: pgmap v187: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:29.530 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:29 smithi161 ceph-mon[32709]: from='client.? 172.21.15.183:0/3894414454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:29 smithi102 ceph-mon[29216]: pgmap v187: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:29 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/3894414454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:31 smithi183 ceph-mon[33137]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:31.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:31 smithi102 ceph-mon[29216]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:31 smithi161 ceph-mon[32709]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:33.328 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:33 smithi183 ceph-mon[33137]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:33.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:33 smithi102 ceph-mon[29216]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:33.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:33 smithi161 ceph-mon[32709]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:35.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:35 smithi183 ceph-mon[33137]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:35.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:35 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:08:35.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:35 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:35.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:35 smithi102 ceph-mon[29216]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:35.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:35 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:08:35.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:35 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:35 smithi161 ceph-mon[32709]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:35 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:08:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:35 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:36.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:36 smithi183 ceph-mon[33137]: Deploying daemon osd.8 on smithi183 2024-04-24T23:08:36.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:36 smithi102 ceph-mon[29216]: Deploying daemon osd.8 on smithi183 2024-04-24T23:08:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:36 smithi161 ceph-mon[32709]: Deploying daemon osd.8 on smithi183 2024-04-24T23:08:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:37 smithi183 ceph-mon[33137]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:37.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:37 smithi102 ceph-mon[29216]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:37 smithi161 ceph-mon[32709]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:39.433 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:39 smithi183 ceph-mon[33137]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:39.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:39 smithi102 ceph-mon[29216]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:39.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:39 smithi161 ceph-mon[32709]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:41.437 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:41 smithi183 ceph-mon[33137]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:41.438 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:41.438 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:41.438 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:41.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:41 smithi102 ceph-mon[29216]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:41.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:41.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:41 smithi161 ceph-mon[32709]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:41.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:41.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:41.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:43.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:43 smithi183 ceph-mon[33137]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:43.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:43 smithi102 ceph-mon[29216]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:43.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:43 smithi161 ceph-mon[32709]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:44.768 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 8 on host 'smithi183' 2024-04-24T23:08:44.768 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.766+0000 7f53a4ff9700 1 -- 172.21.15.183:0/1224550422 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f53a8062670 con 0x7f53900a2ca0 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f53900a2ca0 msgr2=0x7f53900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f53900a2ca0 0x7f53900a5160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53a8069d10 tx=0x7f5398007400 comp rx=0 tx=0).stop 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 msgr2=0x7f53a819f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 0x7f53a819f8c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009850 tx=0x7f53a0004d70 comp rx=0 tx=0).stop 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 shutdown_connections 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a8103ec0 0x7f53a819f8c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53a8100540 0x7f53a8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.770 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f53900a2ca0 0x7f53900a5160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.771 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 --2- 172.21.15.183:0/1224550422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53a8103520 0x7f53a8198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:44.771 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.768+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 >> 172.21.15.183:0/1224550422 conn(0x7f53a80fbec0 msgr2=0x7f53a8106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:44.771 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.769+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 shutdown_connections 2024-04-24T23:08:44.771 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:08:44.769+0000 7f53ada7e700 1 -- 172.21.15.183:0/1224550422 wait complete. 2024-04-24T23:08:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='osd.8 [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:44 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='osd.8 [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:44 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:45.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:45.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:45.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:45.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='osd.8 [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:45.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:08:45.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:45.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:44 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:45.417 DEBUG:teuthology.orchestra.run.smithi183:osd.8> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T23:08:45.419 INFO:tasks.cephadm:Deploying osd.9 on smithi102 with /dev/vg_nvme/lv_1... 2024-04-24T23:08:45.420 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:08:45.665 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:08:45.986 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:45 smithi183 ceph-mon[33137]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:08:45.986 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:45 smithi183 ceph-mon[33137]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:08:45.986 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:45 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:45.986 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:45 smithi183 ceph-mon[33137]: from='osd.8 [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:08:45.987 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:45 smithi183 ceph-mon[33137]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:08:45.987 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:08:45 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8[51784]: 2024-04-24T23:08:45.833+0000 7fe1145ba700 -1 osd.8 0 waiting for initial osdmap 2024-04-24T23:08:45.987 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:08:45 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8[51784]: 2024-04-24T23:08:45.842+0000 7fe1103ad700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:46.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:45 smithi102 ceph-mon[29216]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:08:46.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:45 smithi102 ceph-mon[29216]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:08:46.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:45 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:46.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:45 smithi102 ceph-mon[29216]: from='osd.8 [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:08:46.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:45 smithi102 ceph-mon[29216]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:08:46.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:45 smithi161 ceph-mon[32709]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:08:46.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:45 smithi161 ceph-mon[32709]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:08:46.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:45 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:46.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:45 smithi161 ceph-mon[32709]: from='osd.8 [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:08:46.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:45 smithi161 ceph-mon[32709]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:08:46.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:46 smithi183 ceph-mon[33137]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:08:46.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:46 smithi183 ceph-mon[33137]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T23:08:46.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:46.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:46.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:46 smithi183 ceph-mon[33137]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:47.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:46 smithi161 ceph-mon[32709]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:08:47.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:46 smithi161 ceph-mon[32709]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T23:08:47.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:47.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:47.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:46 smithi161 ceph-mon[32709]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:47.161 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:46 smithi102 ceph-mon[29216]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:08:47.161 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:46 smithi102 ceph-mon[29216]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T23:08:47.161 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:47.161 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:47.161 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:46 smithi102 ceph-mon[29216]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:08:47.904 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:08:47.904 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:08:47.904 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-24T23:08:47.905 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-24T23:08:47.905 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112984 s, 928 MB/s 2024-04-24T23:08:47.905 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-24T23:08:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:47 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:08:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:47 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:08:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:47 smithi183 ceph-mon[33137]: osd.8 [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] boot 2024-04-24T23:08:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:47 smithi183 ceph-mon[33137]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:08:48.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:47 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:08:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:47 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:08:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:47 smithi161 ceph-mon[32709]: osd.8 [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] boot 2024-04-24T23:08:48.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:47 smithi161 ceph-mon[32709]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:08:48.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:48.243 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:47 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:08:48.243 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:47 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:08:48.243 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:48.243 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:47 smithi102 ceph-mon[29216]: osd.8 [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] boot 2024-04-24T23:08:48.243 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:47 smithi102 ceph-mon[29216]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:08:48.244 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:08:48.589 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_1 2024-04-24T23:08:48.832 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:08:49.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:08:49.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:49.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:49.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:49.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:49.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:49.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:49.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:08:49.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:49.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:49.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:49.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:49.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:49.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:49.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:08:49.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:49.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:49.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:49.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:49.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:49.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:49.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:49.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:08:50.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:49 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi183 2024-04-24T23:08:50.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:49 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-24T23:08:50.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:49 smithi102 ceph-mon[29216]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:08:50.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:49 smithi102 ceph-mon[29216]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:08:50.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:49 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi183 2024-04-24T23:08:50.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:49 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-24T23:08:50.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:49 smithi161 ceph-mon[32709]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:08:50.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:49 smithi161 ceph-mon[32709]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:08:50.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:49 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi183 2024-04-24T23:08:50.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:49 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-24T23:08:50.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:49 smithi183 ceph-mon[33137]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:08:50.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:49 smithi183 ceph-mon[33137]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:08:50.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.657+0000 7fbb75d8a700 1 -- 172.21.15.102:0/3004701601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb70100ce0 msgr2=0x7fbb7010a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:50.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.657+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/3004701601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb70100ce0 0x7fbb7010a8b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b0a0 tx=0x7fbb6800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.657+0000 7fbb75d8a700 1 -- 172.21.15.102:0/3004701601 shutdown_connections 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.657+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/3004701601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb70100ce0 0x7fbb7010a8b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.657+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/3004701601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb701038c0 0x7fbb70103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.657+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/3004701601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb70100340 0x7fbb701007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.657+0000 7fbb75d8a700 1 -- 172.21.15.102:0/3004701601 >> 172.21.15.102:0/3004701601 conn(0x7fbb700fbf00 msgr2=0x7fbb700fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.657+0000 7fbb75d8a700 1 -- 172.21.15.102:0/3004701601 shutdown_connections 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.657+0000 7fbb75d8a700 1 -- 172.21.15.102:0/3004701601 wait complete. 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.658+0000 7fbb75d8a700 1 Processor -- start 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.658+0000 7fbb75d8a700 1 -- start start 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.658+0000 7fbb75d8a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb70100340 0x7fbb701982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.658+0000 7fbb75d8a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb70100ce0 0x7fbb701987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.658+0000 7fbb75d8a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb701038c0 0x7fbb7019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.658+0000 7fbb75d8a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7010dd50 con 0x7fbb701038c0 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.658+0000 7fbb75d8a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7010dbd0 con 0x7fbb70100340 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.658+0000 7fbb75d8a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7010ded0 con 0x7fbb70100ce0 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb701038c0 0x7fbb7019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb701038c0 0x7fbb7019f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41326/0 (socket says 172.21.15.102:41326) 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6ffff700 1 -- 172.21.15.102:0/598492086 learned_addr learned my addr 172.21.15.102:0/598492086 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6f7fe700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb70100340 0x7fbb701982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6effd700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb70100ce0 0x7fbb701987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6ffff700 1 -- 172.21.15.102:0/598492086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb70100ce0 msgr2=0x7fbb701987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6ffff700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb70100ce0 0x7fbb701987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6ffff700 1 -- 172.21.15.102:0/598492086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb70100340 msgr2=0x7fbb701982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6ffff700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb70100340 0x7fbb701982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6ffff700 1 -- 172.21.15.102:0/598492086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7019ff00 con 0x7fbb701038c0 2024-04-24T23:08:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6f7fe700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb70100340 0x7fbb701982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6effd700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb70100ce0 0x7fbb701987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6ffff700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb701038c0 0x7fbb7019f860 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb68004d80 tx=0x7fbb68004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6cff9700 1 -- 172.21.15.102:0/598492086 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68021070 con 0x7fbb701038c0 2024-04-24T23:08:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6cff9700 1 -- 172.21.15.102:0/598492086 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb680054f0 con 0x7fbb701038c0 2024-04-24T23:08:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb701a0190 con 0x7fbb701038c0 2024-04-24T23:08:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb6cff9700 1 -- 172.21.15.102:0/598492086 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6801b850 con 0x7fbb701038c0 2024-04-24T23:08:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.659+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb701a05e0 con 0x7fbb701038c0 2024-04-24T23:08:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.660+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb70068f20 con 0x7fbb701038c0 2024-04-24T23:08:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.661+0000 7fbb6cff9700 1 -- 172.21.15.102:0/598492086 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbb680056a0 con 0x7fbb701038c0 2024-04-24T23:08:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.662+0000 7fbb6cff9700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbb580a2c20 0x7fbb580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.662+0000 7fbb6f7fe700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbb580a2c20 0x7fbb580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.662+0000 7fbb6cff9700 1 -- 172.21.15.102:0/598492086 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fbb6800f020 con 0x7fbb701038c0 2024-04-24T23:08:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.662+0000 7fbb6f7fe700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbb580a2c20 0x7fbb580a50e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb7006a6a0 tx=0x7fbb6000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:50.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.667+0000 7fbb6cff9700 1 -- 172.21.15.102:0/598492086 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbb6802a020 con 0x7fbb701038c0 2024-04-24T23:08:50.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:08:50.876+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbb70062670 con 0x7fbb580a2c20 2024-04-24T23:08:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:50 smithi161 ceph-mon[32709]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:50 smithi161 ceph-mon[32709]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T23:08:51.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:50 smithi183 ceph-mon[33137]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:51.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:50 smithi183 ceph-mon[33137]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T23:08:51.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:50 smithi102 ceph-mon[29216]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:51.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:50 smithi102 ceph-mon[29216]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T23:08:52.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:51 smithi102 ceph-mon[29216]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:52.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:52.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:52.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:51 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:52.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:51 smithi161 ceph-mon[32709]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:52.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:52.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:52.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:51 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:52.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:51 smithi183 ceph-mon[33137]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:52.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:52.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:52.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:51 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:53.144 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:52 smithi102 ceph-mon[29216]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:53.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:52 smithi183 ceph-mon[33137]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:53.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:52 smithi161 ceph-mon[32709]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:08:54.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:53 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1331764209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7df24c5-790f-45d6-a588-4de56ecc06d5"}]: dispatch 2024-04-24T23:08:54.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:53 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1331764209' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7df24c5-790f-45d6-a588-4de56ecc06d5"}]': finished 2024-04-24T23:08:54.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:53 smithi183 ceph-mon[33137]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:08:54.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:53 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1331764209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7df24c5-790f-45d6-a588-4de56ecc06d5"}]: dispatch 2024-04-24T23:08:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:53 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1331764209' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7df24c5-790f-45d6-a588-4de56ecc06d5"}]': finished 2024-04-24T23:08:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:53 smithi102 ceph-mon[29216]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:08:54.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:54.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:53 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1331764209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7df24c5-790f-45d6-a588-4de56ecc06d5"}]: dispatch 2024-04-24T23:08:54.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:53 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1331764209' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7df24c5-790f-45d6-a588-4de56ecc06d5"}]': finished 2024-04-24T23:08:54.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:53 smithi161 ceph-mon[32709]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:08:54.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:08:55.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:54 smithi183 ceph-mon[33137]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-24T23:08:55.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:54 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1430510742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:55.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:54 smithi102 ceph-mon[29216]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-24T23:08:55.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:54 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1430510742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:55.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:54 smithi161 ceph-mon[32709]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-24T23:08:55.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:54 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1430510742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:57.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:57 smithi183 ceph-mon[33137]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-24T23:08:57.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:57 smithi102 ceph-mon[29216]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-24T23:08:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:57 smithi161 ceph-mon[32709]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-24T23:08:59.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:08:59 smithi183 ceph-mon[33137]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:08:59.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:08:59 smithi102 ceph-mon[29216]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:08:59.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:08:59 smithi161 ceph-mon[32709]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:09:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:01 smithi183 ceph-mon[33137]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-24T23:09:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:01 smithi102 ceph-mon[29216]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-24T23:09:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:01 smithi161 ceph-mon[32709]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-24T23:09:02.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:09:02.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:02.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:09:02.516 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:02.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:09:02.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:03 smithi183 ceph-mon[33137]: Deploying daemon osd.9 on smithi102 2024-04-24T23:09:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:03 smithi183 ceph-mon[33137]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:09:03.562 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:03 smithi102 ceph-mon[29216]: Deploying daemon osd.9 on smithi102 2024-04-24T23:09:03.562 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:03 smithi102 ceph-mon[29216]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:09:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:03 smithi161 ceph-mon[32709]: Deploying daemon osd.9 on smithi102 2024-04-24T23:09:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:03 smithi161 ceph-mon[32709]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:09:05.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:05 smithi102 ceph-mon[29216]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:05 smithi183 ceph-mon[33137]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:05 smithi161 ceph-mon[32709]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:07.515 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:07 smithi102 ceph-mon[29216]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:07 smithi161 ceph-mon[32709]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:07 smithi183 ceph-mon[33137]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:08.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:08 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:08 smithi102 ceph-mon[29216]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:08 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:08 smithi161 ceph-mon[32709]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:08 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:08 smithi183 ceph-mon[33137]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:11 smithi183 ceph-mon[33137]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:11.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:11 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:11.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:11 smithi102 ceph-mon[29216]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:11.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:11 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:11 smithi161 ceph-mon[32709]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:11.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:11.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:11 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:11.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.739+0000 7fbb6cff9700 1 -- 172.21.15.102:0/598492086 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbb70062670 con 0x7fbb580a2c20 2024-04-24T23:09:11.741 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 9 on host 'smithi102' 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.741+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbb580a2c20 msgr2=0x7fbb580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.741+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbb580a2c20 0x7fbb580a50e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb7006a6a0 tx=0x7fbb6000a680 comp rx=0 tx=0).stop 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb701038c0 msgr2=0x7fbb7019f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb701038c0 0x7fbb7019f860 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb68004d80 tx=0x7fbb68004e60 comp rx=0 tx=0).stop 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 shutdown_connections 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbb580a2c20 0x7fbb580a50e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb701038c0 0x7fbb7019f860 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb70100340 0x7fbb701982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 --2- 172.21.15.102:0/598492086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb70100ce0 0x7fbb701987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 >> 172.21.15.102:0/598492086 conn(0x7fbb700fbf00 msgr2=0x7fbb70106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 shutdown_connections 2024-04-24T23:09:11.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:09:11.742+0000 7fbb75d8a700 1 -- 172.21.15.102:0/598492086 wait complete. 2024-04-24T23:09:12.508 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:12 smithi102 ceph-mon[29216]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:09:12.508 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:12 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:12.508 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:12 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:12.509 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:12 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:12.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:12 smithi183 ceph-mon[33137]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:09:12.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:12 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:12.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:12 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:12.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:12 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:12.535 DEBUG:teuthology.orchestra.run.smithi102:osd.9> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T23:09:12.538 INFO:tasks.cephadm:Deploying osd.10 on smithi161 with /dev/vg_nvme/lv_1... 2024-04-24T23:09:12.538 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:09:12.564 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:12 smithi161 ceph-mon[32709]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:09:12.564 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:12 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:12.565 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:12 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:12.565 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:12 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:12.783 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:09:13.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:13 smithi102 ceph-mon[29216]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:13 smithi102 ceph-mon[29216]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:09:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:13 smithi102 ceph-mon[29216]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:09:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:13 smithi102 ceph-mon[29216]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:09:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:13 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:13 smithi161 ceph-mon[32709]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:13 smithi161 ceph-mon[32709]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:09:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:13 smithi161 ceph-mon[32709]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:09:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:13 smithi161 ceph-mon[32709]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:09:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:13 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:13 smithi183 ceph-mon[33137]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:13 smithi183 ceph-mon[33137]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:09:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:13 smithi183 ceph-mon[33137]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:09:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:13 smithi183 ceph-mon[33137]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-24T23:09:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:13 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:13.822 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:09:13 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:09:13.698+0000 7fa894951700 -1 osd.9 0 waiting for initial osdmap 2024-04-24T23:09:13.822 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:09:13 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:09:13.712+0000 7fa88c73c700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:09:14.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:14 smithi161 ceph-mon[32709]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:09:14.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:14 smithi161 ceph-mon[32709]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T23:09:14.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:14 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:14.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:14 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:14.500 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:14 smithi161 ceph-mon[32709]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' 2024-04-24T23:09:14.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:14 smithi102 ceph-mon[29216]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:09:14.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:14 smithi102 ceph-mon[29216]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T23:09:14.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:14 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:14.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:14 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:14.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:14 smithi102 ceph-mon[29216]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' 2024-04-24T23:09:14.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:14 smithi183 ceph-mon[33137]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-24T23:09:14.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:14 smithi183 ceph-mon[33137]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T23:09:14.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:14 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:14.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:14 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:14.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:14 smithi183 ceph-mon[33137]: from='osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029]' entity='osd.9' 2024-04-24T23:09:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:09:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:09:14.866 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-04-24T23:09:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-04-24T23:09:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114877 s, 913 MB/s 2024-04-24T23:09:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-04-24T23:09:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:15 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:09:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:15 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:09:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:15 smithi102 ceph-mon[29216]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:15.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:15.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:15 smithi102 ceph-mon[29216]: osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] boot 2024-04-24T23:09:15.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:15 smithi102 ceph-mon[29216]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:09:15.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:15 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:09:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:15 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:09:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:15 smithi161 ceph-mon[32709]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:15 smithi161 ceph-mon[32709]: osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] boot 2024-04-24T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:15 smithi161 ceph-mon[32709]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:15.678 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-04-24T23:09:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:15 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:09:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:15 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:09:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:15 smithi183 ceph-mon[33137]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:09:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:15 smithi183 ceph-mon[33137]: osd.9 [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] boot 2024-04-24T23:09:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:15 smithi183 ceph-mon[33137]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:09:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:09:15.969 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:09:17.248 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi102 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi102 to 4272M 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.249 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:17.250 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:16 smithi161 ceph-mon[32709]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T23:09:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi102 2024-04-24T23:09:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi102 to 4272M 2024-04-24T23:09:17.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:17.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:17.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:17.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:16 smithi183 ceph-mon[33137]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T23:09:17.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi102 2024-04-24T23:09:17.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.326 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:17.326 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi102 to 4272M 2024-04-24T23:09:17.326 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.326 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:17.326 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:17.326 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:17.326 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:17.326 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:16 smithi102 ceph-mon[29216]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T23:09:17.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.599+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/3287842920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f01034c0 msgr2=0x7fa5f0103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:17.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/3287842920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f01034c0 0x7fa5f0103920 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800b0d0 tx=0x7fa5e800b3e0 comp rx=0 tx=0).stop 2024-04-24T23:09:17.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/3287842920 shutdown_connections 2024-04-24T23:09:17.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/3287842920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f0103e60 0x7fa5f010a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:17.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/3287842920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5f01022c0 0x7fa5f01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/3287842920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f01034c0 0x7fa5f0103920 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/3287842920 >> 172.21.15.161:0/3287842920 conn(0x7fa5f00fd880 msgr2=0x7fa5f00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/3287842920 shutdown_connections 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/3287842920 wait complete. 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 Processor -- start 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 -- start start 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5f01022c0 0x7fa5f0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f01034c0 0x7fa5f01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0103e60 0x7fa5f019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.600+0000 7fa5f6dc4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f010dd40 con 0x7fa5f01022c0 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f6dc4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f010dbc0 con 0x7fa5f0103e60 2024-04-24T23:09:17.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f6dc4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f010dec0 con 0x7fa5f01034c0 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f5361700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0103e60 0x7fa5f019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f5361700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0103e60 0x7fa5f019f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47350/0 (socket says 172.21.15.161:47350) 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f5361700 1 -- 172.21.15.161:0/160671235 learned_addr learned my addr 172.21.15.161:0/160671235 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f4b60700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5f01022c0 0x7fa5f0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5effff700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f01034c0 0x7fa5f01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f5361700 1 -- 172.21.15.161:0/160671235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f01034c0 msgr2=0x7fa5f01988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f5361700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f01034c0 0x7fa5f01988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f5361700 1 -- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5f01022c0 msgr2=0x7fa5f0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f5361700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5f01022c0 0x7fa5f0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f5361700 1 -- 172.21.15.161:0/160671235 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f019ffe0 con 0x7fa5f0103e60 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f5361700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0103e60 0x7fa5f019f940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400d980 tx=0x7fa5e400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5edffb700 1 -- 172.21.15.161:0/160671235 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4017070 con 0x7fa5f0103e60 2024-04-24T23:09:17.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5edffb700 1 -- 172.21.15.161:0/160671235 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5e4005020 con 0x7fa5f0103e60 2024-04-24T23:09:17.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f00744c0 con 0x7fa5f0103e60 2024-04-24T23:09:17.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.601+0000 7fa5edffb700 1 -- 172.21.15.161:0/160671235 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400fa90 con 0x7fa5f0103e60 2024-04-24T23:09:17.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.602+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f0074a50 con 0x7fa5f0103e60 2024-04-24T23:09:17.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.602+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f0068f20 con 0x7fa5f0103e60 2024-04-24T23:09:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.604+0000 7fa5edffb700 1 -- 172.21.15.161:0/160671235 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa5e40051d0 con 0x7fa5f0103e60 2024-04-24T23:09:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.604+0000 7fa5edffb700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5d80a2ca0 0x7fa5d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.604+0000 7fa5edffb700 1 -- 172.21.15.161:0/160671235 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fa5e40ce9c0 con 0x7fa5f0103e60 2024-04-24T23:09:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.604+0000 7fa5f4b60700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5d80a2ca0 0x7fa5d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:17.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.605+0000 7fa5f4b60700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5d80a2ca0 0x7fa5d80a5160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0103320 tx=0x7fa5e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:17.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.608+0000 7fa5edffb700 1 -- 172.21.15.161:0/160671235 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5e4020020 con 0x7fa5f0103e60 2024-04-24T23:09:17.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:17.810+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa5f0062730 con 0x7fa5d80a2ca0 2024-04-24T23:09:18.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:17 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:18.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:17 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:18.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:17 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:18.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:17 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:18.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:17 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:18.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:17 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:18.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:18.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:18.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:17 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:19.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:18 smithi161 ceph-mon[32709]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:19.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:18 smithi161 ceph-mon[32709]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:19.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:18 smithi183 ceph-mon[33137]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:19.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:18 smithi183 ceph-mon[33137]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:19.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:18 smithi102 ceph-mon[29216]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:19.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:18 smithi102 ceph-mon[29216]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:21.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:21 smithi183 ceph-mon[33137]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:21.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:21 smithi183 ceph-mon[33137]: from='client.? 172.21.15.161:0/552520976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "442f5937-b88f-4000-b961-d097cceb9f88"}]: dispatch 2024-04-24T23:09:21.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:21 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "442f5937-b88f-4000-b961-d097cceb9f88"}]: dispatch 2024-04-24T23:09:21.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:21 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "442f5937-b88f-4000-b961-d097cceb9f88"}]': finished 2024-04-24T23:09:21.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:21 smithi183 ceph-mon[33137]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:09:21.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:21 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:21.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:21 smithi183 ceph-mon[33137]: from='client.? 172.21.15.161:0/2075006261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:21 smithi102 ceph-mon[29216]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:21 smithi102 ceph-mon[29216]: from='client.? 172.21.15.161:0/552520976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "442f5937-b88f-4000-b961-d097cceb9f88"}]: dispatch 2024-04-24T23:09:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:21 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "442f5937-b88f-4000-b961-d097cceb9f88"}]: dispatch 2024-04-24T23:09:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:21 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "442f5937-b88f-4000-b961-d097cceb9f88"}]': finished 2024-04-24T23:09:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:21 smithi102 ceph-mon[29216]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:09:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:21 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:21 smithi102 ceph-mon[29216]: from='client.? 172.21.15.161:0/2075006261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:21.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:21 smithi161 ceph-mon[32709]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:21.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:21 smithi161 ceph-mon[32709]: from='client.? 172.21.15.161:0/552520976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "442f5937-b88f-4000-b961-d097cceb9f88"}]: dispatch 2024-04-24T23:09:21.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:21 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "442f5937-b88f-4000-b961-d097cceb9f88"}]: dispatch 2024-04-24T23:09:21.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:21 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "442f5937-b88f-4000-b961-d097cceb9f88"}]': finished 2024-04-24T23:09:21.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:21 smithi161 ceph-mon[32709]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:09:21.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:21 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:21.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:21 smithi161 ceph-mon[32709]: from='client.? 172.21.15.161:0/2075006261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:23 smithi183 ceph-mon[33137]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:23 smithi102 ceph-mon[29216]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:23 smithi161 ceph-mon[32709]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:25.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:25 smithi183 ceph-mon[33137]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:25.569 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:25 smithi161 ceph-mon[32709]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:25.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:25 smithi102 ceph-mon[29216]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:27 smithi183 ceph-mon[33137]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:27.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:27 smithi102 ceph-mon[29216]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:27 smithi161 ceph-mon[32709]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:09:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:28 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:28 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:09:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:28 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:28 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:09:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:28 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:29.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:29 smithi102 ceph-mon[29216]: Deploying daemon osd.10 on smithi161 2024-04-24T23:09:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:29 smithi102 ceph-mon[29216]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:29.580 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:29 smithi161 ceph-mon[32709]: Deploying daemon osd.10 on smithi161 2024-04-24T23:09:29.580 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:29 smithi161 ceph-mon[32709]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:29 smithi183 ceph-mon[33137]: Deploying daemon osd.10 on smithi161 2024-04-24T23:09:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:29 smithi183 ceph-mon[33137]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:31.540 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:31 smithi161 ceph-mon[32709]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:31.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:31 smithi102 ceph-mon[29216]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:31.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:31 smithi183 ceph-mon[33137]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:33.503 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:33 smithi161 ceph-mon[32709]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:33.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:33 smithi102 ceph-mon[29216]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:33.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:33 smithi183 ceph-mon[33137]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:34.440 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:34 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:34.440 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:34 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:34.440 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:34 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:34.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:34 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:34.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:34 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:34.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:34 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:34.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:34 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:34.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:34 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:34.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:34 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:35.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:35 smithi102 ceph-mon[29216]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:35.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:35 smithi161 ceph-mon[32709]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:35 smithi183 ceph-mon[33137]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:37.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.375+0000 7fa5edffb700 1 -- 172.21.15.161:0/160671235 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa5f0062730 con 0x7fa5d80a2ca0 2024-04-24T23:09:37.378 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 10 on host 'smithi161' 2024-04-24T23:09:37.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5d80a2ca0 msgr2=0x7fa5d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:37.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5d80a2ca0 0x7fa5d80a5160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0103320 tx=0x7fa5e000b410 comp rx=0 tx=0).stop 2024-04-24T23:09:37.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0103e60 msgr2=0x7fa5f019f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0103e60 0x7fa5f019f940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400d980 tx=0x7fa5e400dc90 comp rx=0 tx=0).stop 2024-04-24T23:09:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 shutdown_connections 2024-04-24T23:09:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0103e60 0x7fa5f019f940 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5d80a2ca0 0x7fa5d80a5160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5f01022c0 0x7fa5f0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 --2- 172.21.15.161:0/160671235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f01034c0 0x7fa5f01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 >> 172.21.15.161:0/160671235 conn(0x7fa5f00fd880 msgr2=0x7fa5f0106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 shutdown_connections 2024-04-24T23:09:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-24T23:09:37.379+0000 7fa5f6dc4700 1 -- 172.21.15.161:0/160671235 wait complete. 2024-04-24T23:09:37.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:37 smithi102 ceph-mon[29216]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:37 smithi161 ceph-mon[32709]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:37.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:37 smithi183 ceph-mon[33137]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:38.038 DEBUG:teuthology.orchestra.run.smithi161:osd.10> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T23:09:38.040 INFO:tasks.cephadm:Deploying osd.11 on smithi183 with /dev/vg_nvme/lv_1... 2024-04-24T23:09:38.040 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:09:38.285 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:09:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:38 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:38 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:38 smithi161 ceph-mon[32709]: from='osd.10 [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:09:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:38 smithi161 ceph-mon[32709]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:09:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:38 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:38 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:38 smithi161 ceph-mon[32709]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:38.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:38 smithi183 ceph-mon[33137]: from='osd.10 [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:09:38.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:38 smithi183 ceph-mon[33137]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:09:38.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:38 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.673 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:38 smithi183 ceph-mon[33137]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:38.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:38 smithi102 ceph-mon[29216]: from='osd.10 [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:09:38.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:38 smithi102 ceph-mon[29216]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:09:38.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:38 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:38.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:38 smithi102 ceph-mon[29216]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:39.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:39 smithi183 ceph-mon[33137]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:09:39.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:39 smithi183 ceph-mon[33137]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:09:39.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:39.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:39 smithi183 ceph-mon[33137]: from='osd.10 [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:09:39.715 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:39 smithi183 ceph-mon[33137]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:09:39.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:39 smithi161 ceph-mon[32709]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:09:39.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:39 smithi161 ceph-mon[32709]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:09:39.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:39.754 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:39 smithi161 ceph-mon[32709]: from='osd.10 [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:09:39.755 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:39 smithi161 ceph-mon[32709]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:09:39.755 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:09:39 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:09:39.370+0000 7f05e6a23700 -1 osd.10 0 waiting for initial osdmap 2024-04-24T23:09:39.755 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:09:39 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:09:39.383+0000 7f05e2816700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:09:39.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:39 smithi102 ceph-mon[29216]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:09:39.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:39 smithi102 ceph-mon[29216]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:09:39.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:39.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:39 smithi102 ceph-mon[29216]: from='osd.10 [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:09:39.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:39 smithi102 ceph-mon[29216]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-24T23:09:40.449 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:09:40.449 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:09:40.449 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-24T23:09:40.449 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-24T23:09:40.449 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112496 s, 932 MB/s 2024-04-24T23:09:40.449 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 2024-04-24T23:09:40.449 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-24T23:09:40.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:40 smithi161 ceph-mon[32709]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:09:40.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:40 smithi161 ceph-mon[32709]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T23:09:40.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:40 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:40.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:40 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:40.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:40 smithi161 ceph-mon[32709]: osd.10 [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] boot 2024-04-24T23:09:40.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:40 smithi161 ceph-mon[32709]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:09:40.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:40 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:40.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:40 smithi161 ceph-mon[32709]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:40.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:40 smithi183 ceph-mon[33137]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:09:40.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:40 smithi183 ceph-mon[33137]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T23:09:40.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:40 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:40.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:40 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:40.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:40 smithi183 ceph-mon[33137]: osd.10 [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] boot 2024-04-24T23:09:40.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:40 smithi183 ceph-mon[33137]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:09:40.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:40 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:40.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:40 smithi183 ceph-mon[33137]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:40.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:40 smithi102 ceph-mon[29216]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-24T23:09:40.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:40 smithi102 ceph-mon[29216]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T23:09:40.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:40 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:40.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:40 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:40.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:40 smithi102 ceph-mon[29216]: osd.10 [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] boot 2024-04-24T23:09:40.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:40 smithi102 ceph-mon[29216]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:09:40.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:40 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:09:40.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:40 smithi102 ceph-mon[29216]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:09:41.113 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-04-24T23:09:41.354 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:09:41.604 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:41 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:09:41.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:41 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:09:41.605 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:41 smithi161 ceph-mon[32709]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:09:41.631 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:41 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:09:41.631 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:41 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:09:41.631 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:41 smithi183 ceph-mon[33137]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:09:41.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:41 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:09:41.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:41 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:09:41.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:41 smithi102 ceph-mon[29216]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:09:42.948 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi161 2024-04-24T23:09:42.948 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:42.948 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi161 to 4272M 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:42.949 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi161 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi161 to 4272M 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:09:43.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:43.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:43.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:43.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:43.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi161 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi161 to 4272M 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:43.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:43.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:43.111 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:09:43.178 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.174+0000 7ff3f6587700 1 -- 172.21.15.183:0/460961600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103490 msgr2=0x7ff3f01038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:43.178 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.174+0000 7ff3f6587700 1 --2- 172.21.15.183:0/460961600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103490 0x7ff3f01038f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009990 tx=0x7ff3e4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:09:43.178 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.174+0000 7ff3f6587700 1 -- 172.21.15.183:0/460961600 shutdown_connections 2024-04-24T23:09:43.178 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.174+0000 7ff3f6587700 1 --2- 172.21.15.183:0/460961600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f0103e30 0x7ff3f010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:43.178 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.174+0000 7ff3f6587700 1 --2- 172.21.15.183:0/460961600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f0102290 0x7ff3f01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:43.178 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.174+0000 7ff3f6587700 1 --2- 172.21.15.183:0/460961600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103490 0x7ff3f01038f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.174+0000 7ff3f6587700 1 -- 172.21.15.183:0/460961600 >> 172.21.15.183:0/460961600 conn(0x7ff3f00fd830 msgr2=0x7ff3f00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.174+0000 7ff3f6587700 1 -- 172.21.15.183:0/460961600 shutdown_connections 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.174+0000 7ff3f6587700 1 -- 172.21.15.183:0/460961600 wait complete. 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f6587700 1 Processor -- start 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f6587700 1 -- start start 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f6587700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f0102290 0x7ff3f0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f6587700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f0103490 0x7ff3f0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f6587700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103e30 0x7ff3f019f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f6587700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f010dd20 con 0x7ff3f0102290 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f6587700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f010dba0 con 0x7ff3f0103490 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f6587700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f010dea0 con 0x7ff3f0103e30 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f4b24700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103e30 0x7ff3f019f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:43.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f4b24700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103e30 0x7ff3f019f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:58586/0 (socket says 172.21.15.183:58586) 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.175+0000 7ff3f4b24700 1 -- 172.21.15.183:0/1046018100 learned_addr learned my addr 172.21.15.183:0/1046018100 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3ef7fe700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f0103490 0x7ff3f0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3effff700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f0102290 0x7ff3f0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3f4b24700 1 -- 172.21.15.183:0/1046018100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f0103490 msgr2=0x7ff3f0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3f4b24700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f0103490 0x7ff3f0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3f4b24700 1 -- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f0102290 msgr2=0x7ff3f0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3f4b24700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f0102290 0x7ff3f0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3f4b24700 1 -- 172.21.15.183:0/1046018100 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f01a0030 con 0x7ff3f0103e30 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3effff700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f0102290 0x7ff3f0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3f4b24700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103e30 0x7ff3f019f990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8007f10 tx=0x7ff3e800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3ed7fa700 1 -- 172.21.15.183:0/1046018100 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8004bb0 con 0x7ff3f0103e30 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3ed7fa700 1 -- 172.21.15.183:0/1046018100 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3e8004d50 con 0x7ff3f0103e30 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f01a0320 con 0x7ff3f0103e30 2024-04-24T23:09:43.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3ed7fa700 1 -- 172.21.15.183:0/1046018100 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8005990 con 0x7ff3f0103e30 2024-04-24T23:09:43.181 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.176+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f01a08b0 con 0x7ff3f0103e30 2024-04-24T23:09:43.182 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.177+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f0068f20 con 0x7ff3f0103e30 2024-04-24T23:09:43.182 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.178+0000 7ff3ed7fa700 1 -- 172.21.15.183:0/1046018100 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff3e801a020 con 0x7ff3f0103e30 2024-04-24T23:09:43.182 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.179+0000 7ff3ed7fa700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff3d80a2ca0 0x7ff3d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:43.182 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.179+0000 7ff3ed7fa700 1 -- 172.21.15.183:0/1046018100 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7ff3e80ce780 con 0x7ff3f0103e30 2024-04-24T23:09:43.182 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.179+0000 7ff3effff700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff3d80a2ca0 0x7ff3d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:43.182 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.180+0000 7ff3effff700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff3d80a2ca0 0x7ff3d80a5160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff3f01032f0 tx=0x7ff3e000b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:43.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.183+0000 7ff3ed7fa700 1 -- 172.21.15.183:0/1046018100 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff3e8010070 con 0x7ff3f0103e30 2024-04-24T23:09:43.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:09:43.389+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff3f0062670 con 0x7ff3d80a2ca0 2024-04-24T23:09:44.001 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:43 smithi183 ceph-mon[33137]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:09:44.001 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:43 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:44.001 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:43 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:44.001 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:43 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:44.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:43 smithi102 ceph-mon[29216]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:09:44.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:43 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:44.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:43 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:44.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:43 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:44.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:43 smithi161 ceph-mon[32709]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:09:44.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:43 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:44.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:43 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:44.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:43 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:44 smithi183 ceph-mon[33137]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:44 smithi183 ceph-mon[33137]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T23:09:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:44 smithi183 ceph-mon[33137]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:09:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:44 smithi102 ceph-mon[29216]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:44 smithi102 ceph-mon[29216]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T23:09:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:44 smithi102 ceph-mon[29216]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:09:45.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:44 smithi161 ceph-mon[32709]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:45.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:44 smithi161 ceph-mon[32709]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T23:09:45.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:44 smithi161 ceph-mon[32709]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:09:46.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:46 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/2132883982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46d330b0-0ac0-4bba-b07e-f77fb2a202b8"}]: dispatch 2024-04-24T23:09:46.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:46 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46d330b0-0ac0-4bba-b07e-f77fb2a202b8"}]: dispatch 2024-04-24T23:09:46.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:46 smithi102 ceph-mon[29216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46d330b0-0ac0-4bba-b07e-f77fb2a202b8"}]': finished 2024-04-24T23:09:46.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:46 smithi102 ceph-mon[29216]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:09:46.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:46 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:46.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:46 smithi161 ceph-mon[32709]: from='client.? 172.21.15.183:0/2132883982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46d330b0-0ac0-4bba-b07e-f77fb2a202b8"}]: dispatch 2024-04-24T23:09:46.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:46 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46d330b0-0ac0-4bba-b07e-f77fb2a202b8"}]: dispatch 2024-04-24T23:09:46.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:46 smithi161 ceph-mon[32709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46d330b0-0ac0-4bba-b07e-f77fb2a202b8"}]': finished 2024-04-24T23:09:46.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:46 smithi161 ceph-mon[32709]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:09:46.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:46 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:46.610 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:46 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/2132883982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46d330b0-0ac0-4bba-b07e-f77fb2a202b8"}]: dispatch 2024-04-24T23:09:46.610 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:46 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46d330b0-0ac0-4bba-b07e-f77fb2a202b8"}]: dispatch 2024-04-24T23:09:46.610 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:46 smithi183 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46d330b0-0ac0-4bba-b07e-f77fb2a202b8"}]': finished 2024-04-24T23:09:46.610 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:46 smithi183 ceph-mon[33137]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:09:46.610 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:46 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:09:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:47 smithi102 ceph-mon[29216]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-24T23:09:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:47 smithi102 ceph-mon[29216]: from='client.? 172.21.15.183:0/2653598153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:47 smithi161 ceph-mon[32709]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-24T23:09:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:47 smithi161 ceph-mon[32709]: from='client.? 172.21.15.183:0/2653598153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:47.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:47 smithi183 ceph-mon[33137]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-24T23:09:47.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:47 smithi183 ceph-mon[33137]: from='client.? 172.21.15.183:0/2653598153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:49.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:49 smithi102 ceph-mon[29216]: pgmap v248: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:09:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:49 smithi161 ceph-mon[32709]: pgmap v248: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:09:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:49 smithi183 ceph-mon[33137]: pgmap v248: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:09:51.450 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:51 smithi183 ceph-mon[33137]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-24T23:09:51.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:51 smithi102 ceph-mon[29216]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-24T23:09:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:51 smithi161 ceph-mon[32709]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-24T23:09:53.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:53 smithi183 ceph-mon[33137]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:53.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:53 smithi102 ceph-mon[29216]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:53 smithi161 ceph-mon[32709]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:54.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:09:54.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:54.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:09:54.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:54.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:09:54.672 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:55 smithi102 ceph-mon[29216]: Deploying daemon osd.11 on smithi183 2024-04-24T23:09:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:55 smithi102 ceph-mon[29216]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:55 smithi161 ceph-mon[32709]: Deploying daemon osd.11 on smithi183 2024-04-24T23:09:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:55 smithi161 ceph-mon[32709]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:55.693 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:55 smithi183 ceph-mon[33137]: Deploying daemon osd.11 on smithi183 2024-04-24T23:09:55.693 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:55 smithi183 ceph-mon[33137]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:57.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:57 smithi183 ceph-mon[33137]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:57.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:57 smithi102 ceph-mon[29216]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:57 smithi161 ceph-mon[32709]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:59.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:09:59 smithi161 ceph-mon[32709]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:59.651 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:09:59 smithi183 ceph-mon[33137]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:09:59.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:09:59 smithi102 ceph-mon[29216]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:00.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:00 smithi161 ceph-mon[32709]: overall HEALTH_OK 2024-04-24T23:10:00.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:00.711 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:00 smithi183 ceph-mon[33137]: overall HEALTH_OK 2024-04-24T23:10:00.712 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:00.712 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:00.712 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:00.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:00 smithi102 ceph-mon[29216]: overall HEALTH_OK 2024-04-24T23:10:00.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:00.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:00.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:01.601 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:01 smithi183 ceph-mon[33137]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:01 smithi161 ceph-mon[32709]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:01.723 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:01 smithi102 ceph-mon[29216]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:03 smithi183 ceph-mon[33137]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:03 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:03 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:03 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:03 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:03.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:03 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:03 smithi161 ceph-mon[32709]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:03 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:03 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:03 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:03 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:03 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:03.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:03 smithi102 ceph-mon[29216]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:03.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:03 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:03.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:03 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:03.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:03 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:03.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:03 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:03.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:03 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:03.956 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.953+0000 7ff3ed7fa700 1 -- 172.21.15.183:0/1046018100 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff3f0062670 con 0x7ff3d80a2ca0 2024-04-24T23:10:03.956 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 11 on host 'smithi183' 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff3d80a2ca0 msgr2=0x7ff3d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff3d80a2ca0 0x7ff3d80a5160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff3f01032f0 tx=0x7ff3e000b670 comp rx=0 tx=0).stop 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103e30 msgr2=0x7ff3f019f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103e30 0x7ff3f019f990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8007f10 tx=0x7ff3e800d3b0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 shutdown_connections 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3f0103e30 0x7ff3f019f990 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff3d80a2ca0 0x7ff3d80a5160 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f0102290 0x7ff3f0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 --2- 172.21.15.183:0/1046018100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3f0103490 0x7ff3f0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:03.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 >> 172.21.15.183:0/1046018100 conn(0x7ff3f00fd830 msgr2=0x7ff3f01068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:03.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 shutdown_connections 2024-04-24T23:10:03.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:03.956+0000 7ff3f6587700 1 -- 172.21.15.183:0/1046018100 wait complete. 2024-04-24T23:10:04.647 DEBUG:teuthology.orchestra.run.smithi183:osd.11> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T23:10:04.649 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T23:10:04.650 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T23:10:04.897 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:05.256 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:05.256 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:05.256 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:04 smithi102 ceph-mon[29216]: from='osd.11 [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:10:05.256 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:04 smithi102 ceph-mon[29216]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:10:05.256 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:04 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:05.256 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:04 smithi102 ceph-mon[29216]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:05.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:05.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:05.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:04 smithi183 ceph-mon[33137]: from='osd.11 [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:10:05.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:04 smithi183 ceph-mon[33137]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:10:05.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:04 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:05.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:04 smithi183 ceph-mon[33137]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:04 smithi161 ceph-mon[32709]: from='osd.11 [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:10:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:04 smithi161 ceph-mon[32709]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:10:05.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:04 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:05.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:04 smithi161 ceph-mon[32709]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:06.121 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:05 smithi183 ceph-mon[33137]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:10:06.121 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:05 smithi183 ceph-mon[33137]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:10:06.121 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:05 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:06.121 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:05 smithi183 ceph-mon[33137]: from='osd.11 [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:10:06.122 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:05 smithi183 ceph-mon[33137]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:10:06.272 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:05 smithi102 ceph-mon[29216]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:10:06.272 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:05 smithi102 ceph-mon[29216]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:10:06.272 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:05 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:06.272 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:05 smithi102 ceph-mon[29216]: from='osd.11 [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:10:06.272 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:05 smithi102 ceph-mon[29216]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:10:06.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:05 smithi161 ceph-mon[32709]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:10:06.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:05 smithi161 ceph-mon[32709]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:10:06.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:05 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:06.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:05 smithi161 ceph-mon[32709]: from='osd.11 [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:10:06.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:05 smithi161 ceph-mon[32709]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-24T23:10:06.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.656+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/1141397472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14102280 msgr2=0x7f0d141026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:06.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.656+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/1141397472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14102280 0x7f0d141026a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d040099c0 tx=0x7f0d04009cd0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.656+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/1141397472 shutdown_connections 2024-04-24T23:10:06.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.656+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/1141397472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14103e20 0x7f0d1410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.656+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/1141397472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14102280 0x7f0d141026a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.656+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/1141397472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d14103480 0x7f0d141038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.656+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/1141397472 >> 172.21.15.102:0/1141397472 conn(0x7f0d140fd820 msgr2=0x7f0d140ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.657+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/1141397472 shutdown_connections 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.657+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/1141397472 wait complete. 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.657+0000 7f0d1b2e2700 1 Processor -- start 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.657+0000 7f0d1b2e2700 1 -- start start 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1b2e2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14102280 0x7f0d141983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1b2e2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d14103480 0x7f0d14198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1b2e2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14103e20 0x7f0d1419f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1b2e2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1410dcf0 con 0x7f0d14103e20 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1b2e2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1410db70 con 0x7f0d14103480 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1b2e2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1410de70 con 0x7f0d14102280 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1987f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14103e20 0x7f0d1419f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:06.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1987f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14103e20 0x7f0d1419f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35976/0 (socket says 172.21.15.102:35976) 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1987f700 1 -- 172.21.15.102:0/2205687317 learned_addr learned my addr 172.21.15.102:0/2205687317 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1907e700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14102280 0x7f0d141983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1887d700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d14103480 0x7f0d14198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1987f700 1 -- 172.21.15.102:0/2205687317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14102280 msgr2=0x7f0d141983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1987f700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14102280 0x7f0d141983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1987f700 1 -- 172.21.15.102:0/2205687317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d14103480 msgr2=0x7f0d14198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1987f700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d14103480 0x7f0d14198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1987f700 1 -- 172.21.15.102:0/2205687317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d141a0020 con 0x7f0d14103e20 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1987f700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14103e20 0x7f0d1419f980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c007f10 tx=0x7f0d0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d0a7fc700 1 -- 172.21.15.102:0/2205687317 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c004bb0 con 0x7f0d14103e20 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d141a0310 con 0x7f0d14103e20 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d0a7fc700 1 -- 172.21.15.102:0/2205687317 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d0c004d50 con 0x7f0d14103e20 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d0a7fc700 1 -- 172.21.15.102:0/2205687317 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0058e0 con 0x7f0d14103e20 2024-04-24T23:10:06.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.658+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d141a0850 con 0x7f0d14103e20 2024-04-24T23:10:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.659+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d14068f20 con 0x7f0d14103e20 2024-04-24T23:10:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.660+0000 7f0d0a7fc700 1 -- 172.21.15.102:0/2205687317 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0d0c01a020 con 0x7f0d14103e20 2024-04-24T23:10:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.661+0000 7f0d0a7fc700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0d000a2c20 0x7f0d000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.661+0000 7f0d1907e700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0d000a2c20 0x7f0d000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.661+0000 7f0d0a7fc700 1 -- 172.21.15.102:0/2205687317 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f0d0c0ce760 con 0x7f0d14103e20 2024-04-24T23:10:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.661+0000 7f0d1907e700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0d000a2c20 0x7f0d000a50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000c00 tx=0x7f0d04005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.665+0000 7f0d0a7fc700 1 -- 172.21.15.102:0/2205687317 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0d0c010070 con 0x7f0d14103e20 2024-04-24T23:10:06.762 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:10:06.420+0000 7fd589c45700 -1 osd.11 0 waiting for initial osdmap 2024-04-24T23:10:06.762 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:10:06.434+0000 7fd581a30700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:10:06.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.861+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0d14062e40 con 0x7f0d14103e20 2024-04-24T23:10:06.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.862+0000 7f0d0a7fc700 1 -- 172.21.15.102:0/2205687317 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f0d0c09bc80 con 0x7f0d14103e20 2024-04-24T23:10:06.864 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:06.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0d000a2c20 msgr2=0x7f0d000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:06.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0d000a2c20 0x7f0d000a50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000c00 tx=0x7f0d04005fb0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14103e20 msgr2=0x7f0d1419f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:06.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14103e20 0x7f0d1419f980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c007f10 tx=0x7f0d0c00d3b0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 shutdown_connections 2024-04-24T23:10:06.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0d000a2c20 0x7f0d000a50e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d14103e20 0x7f0d1419f980 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d14102280 0x7f0d141983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 --2- 172.21.15.102:0/2205687317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d14103480 0x7f0d14198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:06.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 >> 172.21.15.102:0/2205687317 conn(0x7f0d140fd820 msgr2=0x7f0d14106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:06.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 shutdown_connections 2024-04-24T23:10:06.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:06.864+0000 7f0d1b2e2700 1 -- 172.21.15.102:0/2205687317 wait complete. 2024-04-24T23:10:07.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:06 smithi102 ceph-mon[29216]: purged_snaps scrub starts 2024-04-24T23:10:07.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:06 smithi102 ceph-mon[29216]: purged_snaps scrub ok 2024-04-24T23:10:07.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:06 smithi102 ceph-mon[29216]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:10:07.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:06 smithi102 ceph-mon[29216]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T23:10:07.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:07.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:06 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:07.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:06 smithi102 ceph-mon[29216]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:07.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:06 smithi102 ceph-mon[29216]: from='osd.11 ' entity='osd.11' 2024-04-24T23:10:07.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:06 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2205687317' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:10:07.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-mon[33137]: purged_snaps scrub starts 2024-04-24T23:10:07.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-mon[33137]: purged_snaps scrub ok 2024-04-24T23:10:07.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-mon[33137]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:10:07.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-mon[33137]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T23:10:07.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:07.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:07.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-mon[33137]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:07.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-mon[33137]: from='osd.11 ' entity='osd.11' 2024-04-24T23:10:07.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:06 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2205687317' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:10:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:06 smithi161 ceph-mon[32709]: purged_snaps scrub starts 2024-04-24T23:10:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:06 smithi161 ceph-mon[32709]: purged_snaps scrub ok 2024-04-24T23:10:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:06 smithi161 ceph-mon[32709]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-24T23:10:07.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:06 smithi161 ceph-mon[32709]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T23:10:07.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:07.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:06 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:07.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:06 smithi161 ceph-mon[32709]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:10:07.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:06 smithi161 ceph-mon[32709]: from='osd.11 ' entity='osd.11' 2024-04-24T23:10:07.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:06 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2205687317' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:10:07.594 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1714000179,"num_in_osds":12,"osd_in_since":1714000186,"num_remapped_pgs":0} 2024-04-24T23:10:08.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: osd.11 [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] boot 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:08.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:07 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: osd.11 [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] boot 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:08.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:08.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:07 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:08.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: osd.11 [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] boot 2024-04-24T23:10:08.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:10:08.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:10:08.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:08.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:08.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:08.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:07 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:08.595 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T23:10:08.837 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:09.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:08 smithi183 ceph-mon[33137]: Detected new or changed devices on smithi183 2024-04-24T23:10:09.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:08 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-24T23:10:09.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:08 smithi183 ceph-mon[33137]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:09.286 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:08 smithi102 ceph-mon[29216]: Detected new or changed devices on smithi183 2024-04-24T23:10:09.286 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:08 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-24T23:10:09.286 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:08 smithi102 ceph-mon[29216]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:09.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:08 smithi161 ceph-mon[32709]: Detected new or changed devices on smithi183 2024-04-24T23:10:09.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:08 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-24T23:10:09.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:08 smithi161 ceph-mon[32709]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:10.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:09 smithi183 ceph-mon[33137]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:10:10.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:09 smithi102 ceph-mon[29216]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:10:10.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:09 smithi161 ceph-mon[32709]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:10:10.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 -- 172.21.15.102:0/606860603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01034c0 msgr2=0x7fc3e0103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 --2- 172.21.15.102:0/606860603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01034c0 0x7fc3e0103920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00099c0 tx=0x7fc3d0009cd0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 -- 172.21.15.102:0/606860603 shutdown_connections 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 --2- 172.21.15.102:0/606860603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e0103e60 0x7fc3e010a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 --2- 172.21.15.102:0/606860603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01022c0 0x7fc3e01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 --2- 172.21.15.102:0/606860603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01034c0 0x7fc3e0103920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 -- 172.21.15.102:0/606860603 >> 172.21.15.102:0/606860603 conn(0x7fc3e00fd880 msgr2=0x7fc3e00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 -- 172.21.15.102:0/606860603 shutdown_connections 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 -- 172.21.15.102:0/606860603 wait complete. 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 Processor -- start 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.584+0000 7fc3e770d700 1 -- start start 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e770d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01022c0 0x7fc3e0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e770d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e770d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e0103e60 0x7fc3e019f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e770d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e010dd50 con 0x7fc3e01022c0 2024-04-24T23:10:10.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e770d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e010dbd0 con 0x7fc3e01034c0 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e770d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e010ded0 con 0x7fc3e0103e60 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e54a9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01022c0 0x7fc3e0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e54a9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01022c0 0x7fc3e0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36002/0 (socket says 172.21.15.102:36002) 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e54a9700 1 -- 172.21.15.102:0/4191006554 learned_addr learned my addr 172.21.15.102:0/4191006554 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e4ca8700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e5caa700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e0103e60 0x7fc3e019f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e54a9700 1 -- 172.21.15.102:0/4191006554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e0103e60 msgr2=0x7fc3e019f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e54a9700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e0103e60 0x7fc3e019f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e54a9700 1 -- 172.21.15.102:0/4191006554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 msgr2=0x7fc3e01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e54a9700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e54a9700 1 -- 172.21.15.102:0/4191006554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e019fff0 con 0x7fc3e01022c0 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e5caa700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e0103e60 0x7fc3e019f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e54a9700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01022c0 0x7fc3e0198390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800d9b0 tx=0x7fc3d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:10.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3d67fc700 1 -- 172.21.15.102:0/4191006554 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8017030 con 0x7fc3e01022c0 2024-04-24T23:10:10.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e01a02e0 con 0x7fc3e01022c0 2024-04-24T23:10:10.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3d67fc700 1 -- 172.21.15.102:0/4191006554 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3d80044d0 con 0x7fc3e01022c0 2024-04-24T23:10:10.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.585+0000 7fc3d67fc700 1 -- 172.21.15.102:0/4191006554 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80039e0 con 0x7fc3e01022c0 2024-04-24T23:10:10.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.586+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e01a0820 con 0x7fc3e01022c0 2024-04-24T23:10:10.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.586+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e0068f20 con 0x7fc3e01022c0 2024-04-24T23:10:10.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.587+0000 7fc3d67fc700 1 -- 172.21.15.102:0/4191006554 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc3d801a070 con 0x7fc3e01022c0 2024-04-24T23:10:10.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.588+0000 7fc3d67fc700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3cc0a2c20 0x7fc3cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:10.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.588+0000 7fc3e4ca8700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3cc0a2c20 0x7fc3cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:10.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.588+0000 7fc3d67fc700 1 -- 172.21.15.102:0/4191006554 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc3d80ce890 con 0x7fc3e01022c0 2024-04-24T23:10:10.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.588+0000 7fc3e4ca8700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3cc0a2c20 0x7fc3cc0a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:10.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.592+0000 7fc3d67fc700 1 -- 172.21.15.102:0/4191006554 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3d809bc90 con 0x7fc3e01022c0 2024-04-24T23:10:10.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.789+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc3e002dbd0 con 0x7fc3e01022c0 2024-04-24T23:10:10.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.790+0000 7fc3d67fc700 1 -- 172.21.15.102:0/4191006554 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7fc3d809b590 con 0x7fc3e01022c0 2024-04-24T23:10:10.791 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3cc0a2c20 msgr2=0x7fc3cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3cc0a2c20 0x7fc3cc0a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d0005fb0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01022c0 msgr2=0x7fc3e0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01022c0 0x7fc3e0198390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800d9b0 tx=0x7fc3d800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 shutdown_connections 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e0103e60 0x7fc3e019f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc3cc0a2c20 0x7fc3cc0a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3e01022c0 0x7fc3e0198390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 --2- 172.21.15.102:0/4191006554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 >> 172.21.15.102:0/4191006554 conn(0x7fc3e00fd880 msgr2=0x7fc3e01067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 shutdown_connections 2024-04-24T23:10:10.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:10.792+0000 7fc3e770d700 1 -- 172.21.15.102:0/4191006554 wait complete. 2024-04-24T23:10:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:10 smithi183 ceph-mon[33137]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:10 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4191006554' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:10:11.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:10 smithi102 ceph-mon[29216]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:11.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:10 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4191006554' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:10:11.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:10 smithi161 ceph-mon[32709]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:11.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:10 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4191006554' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:10:11.593 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1714000207,"num_in_osds":12,"osd_in_since":1714000186,"num_remapped_pgs":0} 2024-04-24T23:10:11.593 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:10:11.829 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:13.393 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:13 smithi102 ceph-mon[29216]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:13 smithi161 ceph-mon[32709]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:13.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.626+0000 7f606c14d700 1 -- 172.21.15.102:0/3100088382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60641071a0 msgr2=0x7f6064107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.626+0000 7f606c14d700 1 --2- 172.21.15.102:0/3100088382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60641071a0 0x7f6064107580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6054009990 tx=0x7f6054009ca0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.626+0000 7f606c14d700 1 -- 172.21.15.102:0/3100088382 shutdown_connections 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.626+0000 7f606c14d700 1 --2- 172.21.15.102:0/3100088382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6064101af0 0x7f606410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.626+0000 7f606c14d700 1 --2- 172.21.15.102:0/3100088382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60641071a0 0x7f6064107580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.626+0000 7f606c14d700 1 --2- 172.21.15.102:0/3100088382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6064101130 0x7f60641015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.626+0000 7f606c14d700 1 -- 172.21.15.102:0/3100088382 >> 172.21.15.102:0/3100088382 conn(0x7f60640780e0 msgr2=0x7f60640784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.626+0000 7f606c14d700 1 -- 172.21.15.102:0/3100088382 shutdown_connections 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.626+0000 7f606c14d700 1 -- 172.21.15.102:0/3100088382 wait complete. 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606c14d700 1 Processor -- start 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606c14d700 1 -- start start 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606c14d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6064101130 0x7f60641975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606c14d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064101af0 0x7f6064197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606c14d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60641071a0 0x7f6064198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606c14d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6064111340 con 0x7f6064101af0 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606c14d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f60641111c0 con 0x7f60641071a0 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606c14d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60641114c0 con 0x7f6064101130 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f60696e8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064101af0 0x7f6064197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f60696e8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064101af0 0x7f6064197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36024/0 (socket says 172.21.15.102:36024) 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f60696e8700 1 -- 172.21.15.102:0/3060061104 learned_addr learned my addr 172.21.15.102:0/3060061104 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606a6ea700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60641071a0 0x7f6064198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f6069ee9700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6064101130 0x7f60641975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f60696e8700 1 -- 172.21.15.102:0/3060061104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6064101130 msgr2=0x7f60641975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f60696e8700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6064101130 0x7f60641975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f60696e8700 1 -- 172.21.15.102:0/3060061104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60641071a0 msgr2=0x7f6064198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f60696e8700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60641071a0 0x7f6064198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f60696e8700 1 -- 172.21.15.102:0/3060061104 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60641a6e60 con 0x7f6064101af0 2024-04-24T23:10:13.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.627+0000 7f606a6ea700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60641071a0 0x7f6064198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:13.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.628+0000 7f60696e8700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064101af0 0x7f6064197b30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f606000c920 tx=0x7f606000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.628+0000 7f605affd700 1 -- 172.21.15.102:0/3060061104 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060007b00 con 0x7f6064101af0 2024-04-24T23:10:13.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.628+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60641a7150 con 0x7f6064101af0 2024-04-24T23:10:13.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.628+0000 7f605affd700 1 -- 172.21.15.102:0/3060061104 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6060004ce0 con 0x7f6064101af0 2024-04-24T23:10:13.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.628+0000 7f605affd700 1 -- 172.21.15.102:0/3060061104 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060005560 con 0x7f6064101af0 2024-04-24T23:10:13.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.628+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60641a76e0 con 0x7f6064101af0 2024-04-24T23:10:13.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.629+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6064068f20 con 0x7f6064101af0 2024-04-24T23:10:13.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.631+0000 7f605affd700 1 -- 172.21.15.102:0/3060061104 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6060004750 con 0x7f6064101af0 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.631+0000 7f605affd700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f60500a2c20 0x7f60500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.631+0000 7f6069ee9700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f60500a2c20 0x7f60500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.631+0000 7f605affd700 1 -- 172.21.15.102:0/3060061104 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f60600cebd0 con 0x7f6064101af0 2024-04-24T23:10:13.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.632+0000 7f6069ee9700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f60500a2c20 0x7f60500a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6054006010 tx=0x7f605401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:13.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.635+0000 7f605affd700 1 -- 172.21.15.102:0/3060061104 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6060014030 con 0x7f6064101af0 2024-04-24T23:10:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:13 smithi183 ceph-mon[33137]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:13.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.832+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f606406a500 con 0x7f6064101af0 2024-04-24T23:10:13.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.833+0000 7f605affd700 1 -- 172.21.15.102:0/3060061104 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18516 (secure 0 0 0) 0x7f606009bfd0 con 0x7f6064101af0 2024-04-24T23:10:13.834 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:13.835 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":82,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:10:08.965212+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:13.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f60500a2c20 msgr2=0x7f60500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f60500a2c20 0x7f60500a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6054006010 tx=0x7f605401a040 comp rx=0 tx=0).stop 2024-04-24T23:10:13.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064101af0 msgr2=0x7f6064197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064101af0 0x7f6064197b30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f606000c920 tx=0x7f606000cce0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 shutdown_connections 2024-04-24T23:10:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60641071a0 0x7f6064198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6064101130 0x7f60641975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f60500a2c20 0x7f60500a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 --2- 172.21.15.102:0/3060061104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064101af0 0x7f6064197b30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.835+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 >> 172.21.15.102:0/3060061104 conn(0x7f60640780e0 msgr2=0x7f60640ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.836+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 shutdown_connections 2024-04-24T23:10:13.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:13.836+0000 7f606c14d700 1 -- 172.21.15.102:0/3060061104 wait complete. 2024-04-24T23:10:14.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:14 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3060061104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:14.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:14 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3060061104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:14.652 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:10:14.652 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T23:10:14.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:14 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3060061104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:14.889 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:15.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:15 smithi102 ceph-mon[29216]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:15 smithi161 ceph-mon[32709]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:15 smithi183 ceph-mon[33137]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.696+0000 7f0cd1415700 1 -- 172.21.15.102:0/1941019945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc101af0 msgr2=0x7f0ccc10dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.696+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1941019945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc101af0 0x7f0ccc10dfe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400b0a0 tx=0x7f0cc400b3b0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.696+0000 7f0cd1415700 1 -- 172.21.15.102:0/1941019945 shutdown_connections 2024-04-24T23:10:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.696+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1941019945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc101af0 0x7f0ccc10dfe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.696+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1941019945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ccc1071a0 0x7f0ccc107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.696+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1941019945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ccc101130 0x7f0ccc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.696+0000 7f0cd1415700 1 -- 172.21.15.102:0/1941019945 >> 172.21.15.102:0/1941019945 conn(0x7f0ccc0780e0 msgr2=0x7f0ccc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.696+0000 7f0cd1415700 1 -- 172.21.15.102:0/1941019945 shutdown_connections 2024-04-24T23:10:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.696+0000 7f0cd1415700 1 -- 172.21.15.102:0/1941019945 wait complete. 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.697+0000 7f0cd1415700 1 Processor -- start 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.697+0000 7f0cd1415700 1 -- start start 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.697+0000 7f0cd1415700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ccc101130 0x7f0ccc19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.697+0000 7f0cd1415700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ccc101af0 0x7f0ccc19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.697+0000 7f0cd1415700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc1071a0 0x7f0ccc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.697+0000 7f0cd1415700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc111340 con 0x7f0ccc1071a0 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.697+0000 7f0cd1415700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1111c0 con 0x7f0ccc101130 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.697+0000 7f0cd1415700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1114c0 con 0x7f0ccc101af0 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.697+0000 7f0ccb7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc1071a0 0x7f0ccc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0ccb7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc1071a0 0x7f0ccc1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59840/0 (socket says 172.21.15.102:59840) 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0ccb7fe700 1 -- 172.21.15.102:0/1325109372 learned_addr learned my addr 172.21.15.102:0/1325109372 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0ccaffd700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ccc101130 0x7f0ccc19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0cc27fc700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ccc101af0 0x7f0ccc19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0ccb7fe700 1 -- 172.21.15.102:0/1325109372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ccc101af0 msgr2=0x7f0ccc19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0ccb7fe700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ccc101af0 0x7f0ccc19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0ccb7fe700 1 -- 172.21.15.102:0/1325109372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ccc101130 msgr2=0x7f0ccc19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0ccb7fe700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ccc101130 0x7f0ccc19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0ccb7fe700 1 -- 172.21.15.102:0/1325109372 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc198fa0 con 0x7f0ccc1071a0 2024-04-24T23:10:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0ccb7fe700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc1071a0 0x7f0ccc1986c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40062a0 tx=0x7f0cc4004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0cc8ff9700 1 -- 172.21.15.102:0/1325109372 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4021070 con 0x7f0ccc1071a0 2024-04-24T23:10:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0cc8ff9700 1 -- 172.21.15.102:0/1325109372 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cc4007c80 con 0x7f0ccc1071a0 2024-04-24T23:10:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0cc8ff9700 1 -- 172.21.15.102:0/1325109372 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc401d450 con 0x7f0ccc1071a0 2024-04-24T23:10:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc199230 con 0x7f0ccc1071a0 2024-04-24T23:10:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.698+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc1a5350 con 0x7f0ccc1071a0 2024-04-24T23:10:16.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.699+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ccc102850 con 0x7f0ccc1071a0 2024-04-24T23:10:16.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.700+0000 7f0cc8ff9700 1 -- 172.21.15.102:0/1325109372 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0cc4003a00 con 0x7f0ccc1071a0 2024-04-24T23:10:16.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.701+0000 7f0cc8ff9700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cb80a2ca0 0x7f0cb80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.701+0000 7f0ccaffd700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cb80a2ca0 0x7f0cb80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.701+0000 7f0cc8ff9700 1 -- 172.21.15.102:0/1325109372 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0cc40cfcd0 con 0x7f0ccc1071a0 2024-04-24T23:10:16.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.701+0000 7f0ccaffd700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cb80a2ca0 0x7f0cb80a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00ac50 tx=0x7f0cbc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.704+0000 7f0cc8ff9700 1 -- 172.21.15.102:0/1325109372 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0cc409d160 con 0x7f0ccc1071a0 2024-04-24T23:10:16.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.899+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ccc1a5520 con 0x7f0ccc1071a0 2024-04-24T23:10:16.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.900+0000 7f0cc8ff9700 1 -- 172.21.15.102:0/1325109372 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f0cc409ca60 con 0x7f0ccc1071a0 2024-04-24T23:10:16.901 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:10:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cb80a2ca0 msgr2=0x7f0cb80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cb80a2ca0 0x7f0cb80a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00ac50 tx=0x7f0cbc00a380 comp rx=0 tx=0).stop 2024-04-24T23:10:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc1071a0 msgr2=0x7f0ccc1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc1071a0 0x7f0ccc1986c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40062a0 tx=0x7f0cc4004e20 comp rx=0 tx=0).stop 2024-04-24T23:10:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 shutdown_connections 2024-04-24T23:10:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cb80a2ca0 0x7f0cb80a5160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ccc1071a0 0x7f0ccc1986c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ccc101130 0x7f0ccc19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 --2- 172.21.15.102:0/1325109372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ccc101af0 0x7f0ccc19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 >> 172.21.15.102:0/1325109372 conn(0x7f0ccc0780e0 msgr2=0x7f0ccc100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 shutdown_connections 2024-04-24T23:10:16.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:16.902+0000 7f0cd1415700 1 -- 172.21.15.102:0/1325109372 wait complete. 2024-04-24T23:10:17.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:17 smithi102 ceph-mon[29216]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:17 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1325109372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:17 smithi161 ceph-mon[32709]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:17 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1325109372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:17.703 INFO:tasks.cephadm:Adding mds.a on smithi102 2024-04-24T23:10:17.703 INFO:tasks.cephadm:Adding mds.d on smithi102 2024-04-24T23:10:17.703 INFO:tasks.cephadm:Adding mds.g on smithi102 2024-04-24T23:10:17.704 INFO:tasks.cephadm:Adding mds.j on smithi102 2024-04-24T23:10:17.704 INFO:tasks.cephadm:Adding mds.b on smithi161 2024-04-24T23:10:17.704 INFO:tasks.cephadm:Adding mds.e on smithi161 2024-04-24T23:10:17.704 INFO:tasks.cephadm:Adding mds.h on smithi161 2024-04-24T23:10:17.704 INFO:tasks.cephadm:Adding mds.k on smithi161 2024-04-24T23:10:17.704 INFO:tasks.cephadm:Adding mds.c on smithi183 2024-04-24T23:10:17.704 INFO:tasks.cephadm:Adding mds.f on smithi183 2024-04-24T23:10:17.704 INFO:tasks.cephadm:Adding mds.i on smithi183 2024-04-24T23:10:17.704 INFO:tasks.cephadm:Adding mds.l on smithi183 2024-04-24T23:10:17.704 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi102=a;smithi102=d;smithi102=g;smithi102=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l' 2024-04-24T23:10:17.731 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:17 smithi183 ceph-mon[33137]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:17.731 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:17 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1325109372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:17.953 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:10:19.400 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:19 smithi183 ceph-mon[33137]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:19 smithi102 ceph-mon[29216]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:19 smithi161 ceph-mon[32709]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:19.615 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.613+0000 7f092f743700 1 -- 172.21.15.183:0/1824177852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928102290 msgr2=0x7f09281026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.615 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.613+0000 7f092f743700 1 --2- 172.21.15.183:0/1824177852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928102290 0x7f09281026b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f092000b0a0 tx=0x7f092000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.615 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.613+0000 7f092f743700 1 -- 172.21.15.183:0/1824177852 shutdown_connections 2024-04-24T23:10:19.615 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.613+0000 7f092f743700 1 --2- 172.21.15.183:0/1824177852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0928103e30 0x7f092810a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.615 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.613+0000 7f092f743700 1 --2- 172.21.15.183:0/1824177852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928102290 0x7f09281026b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.615 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.613+0000 7f092f743700 1 --2- 172.21.15.183:0/1824177852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0928103490 0x7f09281038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.615 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.613+0000 7f092f743700 1 -- 172.21.15.183:0/1824177852 >> 172.21.15.183:0/1824177852 conn(0x7f09280fd830 msgr2=0x7f09280ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.615 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.613+0000 7f092f743700 1 -- 172.21.15.183:0/1824177852 shutdown_connections 2024-04-24T23:10:19.616 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.613+0000 7f092f743700 1 -- 172.21.15.183:0/1824177852 wait complete. 2024-04-24T23:10:19.617 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092f743700 1 Processor -- start 2024-04-24T23:10:19.617 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092f743700 1 -- start start 2024-04-24T23:10:19.617 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092f743700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0928102290 0x7f0928198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092f743700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0928103490 0x7f09281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092f743700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928103e30 0x7f092819f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092f743700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f092810dd30 con 0x7f0928103490 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092f743700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f092810dbb0 con 0x7f0928102290 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092f743700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f092810deb0 con 0x7f0928103e30 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092dce0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928103e30 0x7f092819f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092dce0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928103e30 0x7f092819f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57474/0 (socket says 172.21.15.183:57474) 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.614+0000 7f092dce0700 1 -- 172.21.15.183:0/49454682 learned_addr learned my addr 172.21.15.183:0/49454682 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092ccde700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0928103490 0x7f09281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092d4df700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0928102290 0x7f0928198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092dce0700 1 -- 172.21.15.183:0/49454682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0928102290 msgr2=0x7f0928198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092dce0700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0928102290 0x7f0928198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092dce0700 1 -- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0928103490 msgr2=0x7f09281989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.618 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092dce0700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0928103490 0x7f09281989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092dce0700 1 -- 172.21.15.183:0/49454682 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09281a0070 con 0x7f0928103e30 2024-04-24T23:10:19.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092ccde700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0928103490 0x7f09281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:19.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092dce0700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928103e30 0x7f092819f9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f092400d9b0 tx=0x7f092400dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f091e7fc700 1 -- 172.21.15.183:0/49454682 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924017070 con 0x7f0928103e30 2024-04-24T23:10:19.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09281a0360 con 0x7f0928103e30 2024-04-24T23:10:19.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f091e7fc700 1 -- 172.21.15.183:0/49454682 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0924009df0 con 0x7f0928103e30 2024-04-24T23:10:19.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f091e7fc700 1 -- 172.21.15.183:0/49454682 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240053a0 con 0x7f0928103e30 2024-04-24T23:10:19.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.615+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09281a08a0 con 0x7f0928103e30 2024-04-24T23:10:19.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.616+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0928068f20 con 0x7f0928103e30 2024-04-24T23:10:19.620 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.617+0000 7f091e7fc700 1 -- 172.21.15.183:0/49454682 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0924004750 con 0x7f0928103e30 2024-04-24T23:10:19.620 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.618+0000 7f091e7fc700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f09140a2c20 0x7f09140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.620 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.618+0000 7f091e7fc700 1 -- 172.21.15.183:0/49454682 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09240ceee0 con 0x7f0928103e30 2024-04-24T23:10:19.620 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.618+0000 7f092d4df700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f09140a2c20 0x7f09140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.620 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.618+0000 7f092d4df700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f09140a2c20 0x7f09140a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f092000b070 tx=0x7f09200060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.623 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.621+0000 7f091e7fc700 1 -- 172.21.15.183:0/49454682 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0924020020 con 0x7f0928103e30 2024-04-24T23:10:19.830 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.828+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi102=a;smithi102=d;smithi102=g;smithi102=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}) -- 0x7f0928062930 con 0x7f09140a2c20 2024-04-24T23:10:19.838 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.836+0000 7f091e7fc700 1 -- 172.21.15.183:0/49454682 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0928062930 con 0x7f09140a2c20 2024-04-24T23:10:19.838 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mds.all update... 2024-04-24T23:10:19.840 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f09140a2c20 msgr2=0x7f09140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.840 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f09140a2c20 0x7f09140a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f092000b070 tx=0x7f09200060c0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.840 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928103e30 msgr2=0x7f092819f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.840 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928103e30 0x7f092819f9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f092400d9b0 tx=0x7f092400dd70 comp rx=0 tx=0).stop 2024-04-24T23:10:19.840 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 shutdown_connections 2024-04-24T23:10:19.841 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0928103e30 0x7f092819f9d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.841 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0928102290 0x7f0928198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.841 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f09140a2c20 0x7f09140a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.841 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 --2- 172.21.15.183:0/49454682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0928103490 0x7f09281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.841 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.838+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 >> 172.21.15.183:0/49454682 conn(0x7f09280fd830 msgr2=0x7f09281068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.841 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.839+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 shutdown_connections 2024-04-24T23:10:19.841 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-24T23:10:19.839+0000 7f092f743700 1 -- 172.21.15.183:0/49454682 wait complete. 2024-04-24T23:10:20.517 DEBUG:teuthology.orchestra.run.smithi102:mds.a> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T23:10:20.520 DEBUG:teuthology.orchestra.run.smithi102:mds.d> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T23:10:20.563 DEBUG:teuthology.orchestra.run.smithi102:mds.g> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T23:10:20.606 DEBUG:teuthology.orchestra.run.smithi102:mds.j> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T23:10:20.649 DEBUG:teuthology.orchestra.run.smithi161:mds.b> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T23:10:20.651 DEBUG:teuthology.orchestra.run.smithi161:mds.e> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T23:10:20.693 DEBUG:teuthology.orchestra.run.smithi161:mds.h> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T23:10:20.736 DEBUG:teuthology.orchestra.run.smithi161:mds.k> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T23:10:20.779 DEBUG:teuthology.orchestra.run.smithi183:mds.c> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T23:10:20.782 DEBUG:teuthology.orchestra.run.smithi183:mds.f> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T23:10:20.826 DEBUG:teuthology.orchestra.run.smithi183:mds.i> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T23:10:20.868 DEBUG:teuthology.orchestra.run.smithi183:mds.l> sudo journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T23:10:20.911 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T23:10:20.912 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 -- 172.21.15.102:0/1953389705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8102980 msgr2=0x7fdfb8102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 --2- 172.21.15.102:0/1953389705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8102980 0x7fdfb8102e00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009900 tx=0x7fdfac02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 -- 172.21.15.102:0/1953389705 shutdown_connections 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 --2- 172.21.15.102:0/1953389705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb8103340 0x7fdfb8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 --2- 172.21.15.102:0/1953389705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8102980 0x7fdfb8102e00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 --2- 172.21.15.102:0/1953389705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfb8101790 0x7fdfb8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 -- 172.21.15.102:0/1953389705 >> 172.21.15.102:0/1953389705 conn(0x7fdfb80fcd20 msgr2=0x7fdfb80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 -- 172.21.15.102:0/1953389705 shutdown_connections 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 -- 172.21.15.102:0/1953389705 wait complete. 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 Processor -- start 2024-04-24T23:10:21.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 -- start start 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8101790 0x7fdfb8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfb8102980 0x7fdfb819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb8103340 0x7fdfb81a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb810d2a0 con 0x7fdfb8101790 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb810d120 con 0x7fdfb8102980 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.019+0000 7fdfc045d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb810d420 con 0x7fdfb8103340 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe1f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8101790 0x7fdfb8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe1f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8101790 0x7fdfb8199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59870/0 (socket says 172.21.15.102:59870) 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe1f9640 1 -- 172.21.15.102:0/230483772 learned_addr learned my addr 172.21.15.102:0/230483772 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:21.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe9fa640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb8103340 0x7fdfb81a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbd9f8640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfb8102980 0x7fdfb819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe1f9640 1 -- 172.21.15.102:0/230483772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb8103340 msgr2=0x7fdfb81a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe1f9640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb8103340 0x7fdfb81a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe1f9640 1 -- 172.21.15.102:0/230483772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfb8102980 msgr2=0x7fdfb819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe1f9640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfb8102980 0x7fdfb819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe1f9640 1 -- 172.21.15.102:0/230483772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb81a17d0 con 0x7fdfb8101790 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbd9f8640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfb8102980 0x7fdfb819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfbe1f9640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8101790 0x7fdfb8199bb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800ccb0 tx=0x7fdfa800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfb77fe640 1 -- 172.21.15.102:0/230483772 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8003b70 con 0x7fdfb8101790 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfb77fe640 1 -- 172.21.15.102:0/230483772 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfa800ae50 con 0x7fdfb8101790 2024-04-24T23:10:21.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb81a1ac0 con 0x7fdfb8101790 2024-04-24T23:10:21.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfb77fe640 1 -- 172.21.15.102:0/230483772 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800f6f0 con 0x7fdfb8101790 2024-04-24T23:10:21.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.020+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb81a2010 con 0x7fdfb8101790 2024-04-24T23:10:21.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.021+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb81a2240 con 0x7fdfb8101790 2024-04-24T23:10:21.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.022+0000 7fdfb77fe640 1 -- 172.21.15.102:0/230483772 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdfa8004000 con 0x7fdfb8101790 2024-04-24T23:10:21.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.023+0000 7fdfb77fe640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdfa00a2b90 0x7fdfa00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.023+0000 7fdfbd9f8640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdfa00a2b90 0x7fdfa00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.023+0000 7fdfb77fe640 1 -- 172.21.15.102:0/230483772 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdfa80ced60 con 0x7fdfb8101790 2024-04-24T23:10:21.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.023+0000 7fdfbd9f8640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdfa00a2b90 0x7fdfa00a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006010 tx=0x7fdfac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.027+0000 7fdfb77fe640 1 -- 172.21.15.102:0/230483772 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdfa8014070 con 0x7fdfb8101790 2024-04-24T23:10:21.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi102=a;smithi102=d;smithi102=g;smithi102=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:21.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: Saving service mds.all spec with placement smithi102=a;smithi102=d;smithi102=g;smithi102=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-04-24T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:20 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:21.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.208+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfb81a2410 con 0x7fdfb8101790 2024-04-24T23:10:21.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.209+0000 7fdfb77fe640 1 -- 172.21.15.102:0/230483772 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18516 (secure 0 0 0) 0x7fdfa809c160 con 0x7fdfb8101790 2024-04-24T23:10:21.211 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:21.211 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":82,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:10:08.965212+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:21.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdfa00a2b90 msgr2=0x7fdfa00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdfa00a2b90 0x7fdfa00a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006010 tx=0x7fdfac000bc0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8101790 msgr2=0x7fdfb8199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8101790 0x7fdfb8199bb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800ccb0 tx=0x7fdfa800a620 comp rx=0 tx=0).stop 2024-04-24T23:10:21.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 shutdown_connections 2024-04-24T23:10:21.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdfa00a2b90 0x7fdfa00a5050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb8103340 0x7fdfb81a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfb8102980 0x7fdfb819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 --2- 172.21.15.102:0/230483772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfb8101790 0x7fdfb8199bb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 >> 172.21.15.102:0/230483772 conn(0x7fdfb80fcd20 msgr2=0x7fdfb8073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 shutdown_connections 2024-04-24T23:10:21.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.211+0000 7fdfc045d640 1 -- 172.21.15.102:0/230483772 wait complete. 2024-04-24T23:10:21.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi102=a;smithi102=d;smithi102=g;smithi102=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:21.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: Saving service mds.all spec with placement smithi102=a;smithi102=d;smithi102=g;smithi102=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-04-24T23:10:21.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:21.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:21.223 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:21.224 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:21.224 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:21.224 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:21.224 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:21.224 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:21.224 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:20 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:21.225 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:10:21.225 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi102=a;smithi102=d;smithi102=g;smithi102=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: Saving service mds.all spec with placement smithi102=a;smithi102=d;smithi102=g;smithi102=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-04-24T23:10:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:21.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:21.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:21.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:20 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.324+0000 7f4890fe7640 1 -- 172.21.15.102:0/391625489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c101870 msgr2=0x7f488c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.324+0000 7f4890fe7640 1 --2- 172.21.15.102:0/391625489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c101870 0x7f488c101cf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4880009900 tx=0x7f488002a410 comp rx=0 tx=0).stop 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.324+0000 7f4890fe7640 1 -- 172.21.15.102:0/391625489 shutdown_connections 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.324+0000 7f4890fe7640 1 --2- 172.21.15.102:0/391625489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f488c102230 0x7f488c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.324+0000 7f4890fe7640 1 --2- 172.21.15.102:0/391625489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c101870 0x7f488c101cf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.324+0000 7f4890fe7640 1 --2- 172.21.15.102:0/391625489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c100680 0x7f488c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.324+0000 7f4890fe7640 1 -- 172.21.15.102:0/391625489 >> 172.21.15.102:0/391625489 conn(0x7f488c0fbbf0 msgr2=0x7f488c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.324+0000 7f4890fe7640 1 -- 172.21.15.102:0/391625489 shutdown_connections 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.324+0000 7f4890fe7640 1 -- 172.21.15.102:0/391625489 wait complete. 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f4890fe7640 1 Processor -- start 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f4890fe7640 1 -- start start 2024-04-24T23:10:21.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f4890fe7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c100680 0x7f488c197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f4890fe7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f488c101870 0x7f488c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f4890fe7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c102230 0x7f488c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f4890fe7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f488c10c1b0 con 0x7f488c102230 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f4890fe7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f488c10c030 con 0x7f488c101870 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f4890fe7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f488c10c330 con 0x7f488c100680 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f488ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c102230 0x7f488c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f488ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c102230 0x7f488c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59888/0 (socket says 172.21.15.102:59888) 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f488ad9d640 1 -- 172.21.15.102:0/2088993187 learned_addr learned my addr 172.21.15.102:0/2088993187 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f488a59c640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c100680 0x7f488c197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.325+0000 7f4889d9b640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f488c101870 0x7f488c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f488ad9d640 1 -- 172.21.15.102:0/2088993187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c100680 msgr2=0x7f488c197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f488ad9d640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c100680 0x7f488c197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f488ad9d640 1 -- 172.21.15.102:0/2088993187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f488c101870 msgr2=0x7f488c197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f488ad9d640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f488c101870 0x7f488c197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f488ad9d640 1 -- 172.21.15.102:0/2088993187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c19f640 con 0x7f488c102230 2024-04-24T23:10:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f4889d9b640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f488c101870 0x7f488c197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f488ad9d640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c102230 0x7f488c19efd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f487c00d950 tx=0x7f487c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f487b7fe640 1 -- 172.21.15.102:0/2088993187 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c017030 con 0x7f488c102230 2024-04-24T23:10:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f487b7fe640 1 -- 172.21.15.102:0/2088993187 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f487c004020 con 0x7f488c102230 2024-04-24T23:10:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488c19f930 con 0x7f488c102230 2024-04-24T23:10:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f487b7fe640 1 -- 172.21.15.102:0/2088993187 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c003b20 con 0x7f488c102230 2024-04-24T23:10:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.326+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488c19fe80 con 0x7f488c102230 2024-04-24T23:10:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.327+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c1a0110 con 0x7f488c102230 2024-04-24T23:10:21.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.328+0000 7f487b7fe640 1 -- 172.21.15.102:0/2088993187 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f487c01a070 con 0x7f488c102230 2024-04-24T23:10:21.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.329+0000 7f487b7fe640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f48740a2c10 0x7f48740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.329+0000 7f488a59c640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f48740a2c10 0x7f48740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.329+0000 7f487b7fe640 1 -- 172.21.15.102:0/2088993187 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f487c0ce370 con 0x7f488c102230 2024-04-24T23:10:21.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.329+0000 7f488a59c640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f48740a2c10 0x7f48740a50d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f488c1016d0 tx=0x7f4884009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.333+0000 7f487b7fe640 1 -- 172.21.15.102:0/2088993187 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f487c021070 con 0x7f488c102230 2024-04-24T23:10:21.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.506+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f488c1a02e0 con 0x7f488c102230 2024-04-24T23:10:21.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.506+0000 7f487b7fe640 1 -- 172.21.15.102:0/2088993187 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f487c09b770 con 0x7f488c102230 2024-04-24T23:10:21.508 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.508+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f48740a2c10 msgr2=0x7f48740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.508+0000 7f4890fe7640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f48740a2c10 0x7f48740a50d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f488c1016d0 tx=0x7f4884009500 comp rx=0 tx=0).stop 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.508+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c102230 msgr2=0x7f488c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.508+0000 7f4890fe7640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c102230 0x7f488c19efd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f487c00d950 tx=0x7f487c00de10 comp rx=0 tx=0).stop 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.509+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 shutdown_connections 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.509+0000 7f4890fe7640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f48740a2c10 0x7f48740a50d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.509+0000 7f4890fe7640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f488c102230 0x7f488c19efd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.509+0000 7f4890fe7640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f488c101870 0x7f488c197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.509+0000 7f4890fe7640 1 --2- 172.21.15.102:0/2088993187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c100680 0x7f488c197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.509+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 >> 172.21.15.102:0/2088993187 conn(0x7f488c0fbbf0 msgr2=0x7f488c069dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.509+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 shutdown_connections 2024-04-24T23:10:21.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.509+0000 7f4890fe7640 1 -- 172.21.15.102:0/2088993187 wait complete. 2024-04-24T23:10:21.522 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 -- 172.21.15.102:0/2592180495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04100690 msgr2=0x7f2c04100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 --2- 172.21.15.102:0/2592180495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04100690 0x7f2c04100ab0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009900 tx=0x7f2bf402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 -- 172.21.15.102:0/2592180495 shutdown_connections 2024-04-24T23:10:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 --2- 172.21.15.102:0/2592180495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c04102240 0x7f2c04108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 --2- 172.21.15.102:0/2592180495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04101880 0x7f2c04101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 --2- 172.21.15.102:0/2592180495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04100690 0x7f2c04100ab0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 -- 172.21.15.102:0/2592180495 >> 172.21.15.102:0/2592180495 conn(0x7f2c040fbc20 msgr2=0x7f2c040fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 -- 172.21.15.102:0/2592180495 shutdown_connections 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 -- 172.21.15.102:0/2592180495 wait complete. 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 Processor -- start 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.618+0000 7f2c0b970640 1 -- start start 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c0b970640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04100690 0x7f2c04197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c0b970640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04101880 0x7f2c04197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c0b970640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c04102240 0x7f2c0419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c0b970640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0410d2e0 con 0x7f2c04101880 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c0b970640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0410d160 con 0x7f2c04100690 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c0b970640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0410d460 con 0x7f2c04102240 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c08f0b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04101880 0x7f2c04197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c08f0b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04101880 0x7f2c04197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59906/0 (socket says 172.21.15.102:59906) 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c08f0b640 1 -- 172.21.15.102:0/3728285042 learned_addr learned my addr 172.21.15.102:0/3728285042 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c09f0d640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c04102240 0x7f2c0419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c0970c640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04100690 0x7f2c04197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c08f0b640 1 -- 172.21.15.102:0/3728285042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c04102240 msgr2=0x7f2c0419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c08f0b640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c04102240 0x7f2c0419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c08f0b640 1 -- 172.21.15.102:0/3728285042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04100690 msgr2=0x7f2c04197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c08f0b640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04100690 0x7f2c04197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c08f0b640 1 -- 172.21.15.102:0/3728285042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0419f5a0 con 0x7f2c04101880 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c08f0b640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04101880 0x7f2c04197ec0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c920 tx=0x7f2c0000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2bfa7fc640 1 -- 172.21.15.102:0/3728285042 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00007c70 con 0x7f2c04101880 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2bfa7fc640 1 -- 172.21.15.102:0/3728285042 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c00007e10 con 0x7f2c04101880 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0419f890 con 0x7f2c04101880 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2bfa7fc640 1 -- 172.21.15.102:0/3728285042 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00018820 con 0x7f2c04101880 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.619+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0419fde0 con 0x7f2c04101880 2024-04-24T23:10:21.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.620+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c041079d0 con 0x7f2c04101880 2024-04-24T23:10:21.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.621+0000 7f2bfa7fc640 1 -- 172.21.15.102:0/3728285042 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2c000189c0 con 0x7f2c04101880 2024-04-24T23:10:21.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.622+0000 7f2bfa7fc640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bf00a2b90 0x7f2bf00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:21.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.622+0000 7f2c0970c640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bf00a2b90 0x7f2bf00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:21.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.622+0000 7f2bfa7fc640 1 -- 172.21.15.102:0/3728285042 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2c000cecd0 con 0x7f2c04101880 2024-04-24T23:10:21.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.622+0000 7f2c0970c640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bf00a2b90 0x7f2bf00a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4000c00 tx=0x7f2bf40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:21.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.626+0000 7f2bfa7fc640 1 -- 172.21.15.102:0/3728285042 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c00014030 con 0x7f2c04101880 2024-04-24T23:10:21.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.839+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c0402d170 con 0x7f2c04101880 2024-04-24T23:10:21.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.839+0000 7f2bfa7fc640 1 -- 172.21.15.102:0/3728285042 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f2c0009c0d0 con 0x7f2c04101880 2024-04-24T23:10:21.841 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:21.841 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T23:10:21.842 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 1 2024-04-24T23:10:21.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bf00a2b90 msgr2=0x7f2bf00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bf00a2b90 0x7f2bf00a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4000c00 tx=0x7f2bf40314a0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04101880 msgr2=0x7f2c04197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:21.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04101880 0x7f2c04197ec0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c920 tx=0x7f2c0000cde0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 shutdown_connections 2024-04-24T23:10:21.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bf00a2b90 0x7f2bf00a5050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c04102240 0x7f2c0419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c04101880 0x7f2c04197ec0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 --2- 172.21.15.102:0/3728285042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c04100690 0x7f2c04197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:21.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 >> 172.21.15.102:0/3728285042 conn(0x7f2c040fbc20 msgr2=0x7f2c04104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:21.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 shutdown_connections 2024-04-24T23:10:21.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:21.842+0000 7f2c0b970640 1 -- 172.21.15.102:0/3728285042 wait complete. 2024-04-24T23:10:22.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:21 smithi183 ceph-mon[33137]: Deploying daemon mds.l on smithi183 2024-04-24T23:10:22.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:21 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/230483772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:22.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:21 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2088993187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:22.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:21 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3728285042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:22.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:21 smithi161 ceph-mon[32709]: Deploying daemon mds.l on smithi183 2024-04-24T23:10:22.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:21 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/230483772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:22.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:21 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2088993187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:22.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:21 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3728285042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:22.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:21 smithi102 ceph-mon[29216]: Deploying daemon mds.l on smithi183 2024-04-24T23:10:22.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:21 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/230483772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:22.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:21 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2088993187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:22.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:21 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3728285042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:23.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:22 smithi183 ceph-mon[33137]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:23.012 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:10:22 smithi183 systemd[1]: Starting Ceph mds.l for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:23.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:22 smithi161 ceph-mon[32709]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:23.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:22 smithi102 ceph-mon[29216]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:23.352 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:10:23 smithi183 podman[62747]: 2024-04-24 23:10:22.972864777 +0000 UTC m=+0.018185014 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:23.352 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:10:23 smithi183 podman[62747]: 2024-04-24 23:10:23.115771644 +0000 UTC m=+0.161091871 container create a897049076633700cfbd76d084a1fff148997de49be7ccf5c3410b4a37703b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T23:10:23.352 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:10:23 smithi183 podman[62747]: 2024-04-24 23:10:23.351657669 +0000 UTC m=+0.396977896 container init a897049076633700cfbd76d084a1fff148997de49be7ccf5c3410b4a37703b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-24T23:10:23.762 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:10:23 smithi183 podman[62747]: 2024-04-24 23:10:23.354173772 +0000 UTC m=+0.399494004 container start a897049076633700cfbd76d084a1fff148997de49be7ccf5c3410b4a37703b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-04-24T23:10:23.762 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:10:23 smithi183 ceph-mds[62766]: -- 172.21.15.183:0/3594643949 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633dbb8da40 con 0x5633dbb60800 2024-04-24T23:10:23.762 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:10:23 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-l[62762]: starting mds.l at 2024-04-24T23:10:23.762 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:10:23 smithi183 bash[62747]: a897049076633700cfbd76d084a1fff148997de49be7ccf5c3410b4a37703b91 2024-04-24T23:10:23.763 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:10:23 smithi183 systemd[1]: Started Ceph mds.l for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:23.855 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.967+0000 7fc862f1d640 1 -- 172.21.15.102:0/3515500302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c1007b0 msgr2=0x7fc85c100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.967+0000 7fc862f1d640 1 --2- 172.21.15.102:0/3515500302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c1007b0 0x7fc85c100c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc848009900 tx=0x7fc84802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.967+0000 7fc862f1d640 1 -- 172.21.15.102:0/3515500302 shutdown_connections 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.967+0000 7fc862f1d640 1 --2- 172.21.15.102:0/3515500302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c101170 0x7fc85c105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.967+0000 7fc862f1d640 1 --2- 172.21.15.102:0/3515500302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c1007b0 0x7fc85c100c30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.967+0000 7fc862f1d640 1 --2- 172.21.15.102:0/3515500302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc85c0ffa50 0x7fc85c0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.967+0000 7fc862f1d640 1 -- 172.21.15.102:0/3515500302 >> 172.21.15.102:0/3515500302 conn(0x7fc85c0fb380 msgr2=0x7fc85c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.967+0000 7fc862f1d640 1 -- 172.21.15.102:0/3515500302 shutdown_connections 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.967+0000 7fc862f1d640 1 -- 172.21.15.102:0/3515500302 wait complete. 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc862f1d640 1 Processor -- start 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc862f1d640 1 -- start start 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc862f1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ffa50 0x7fc85c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc862f1d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c1007b0 0x7fc85c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc862f1d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc85c101170 0x7fc85c1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc862f1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c113930 con 0x7fc85c0ffa50 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc862f1d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1137b0 con 0x7fc85c101170 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc862f1d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c113ab0 con 0x7fc85c1007b0 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc860cb9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ffa50 0x7fc85c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc860cb9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ffa50 0x7fc85c19e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59928/0 (socket says 172.21.15.102:59928) 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc85bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c1007b0 0x7fc85c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc860cb9640 1 -- 172.21.15.102:0/1843047714 learned_addr learned my addr 172.21.15.102:0/1843047714 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.968+0000 7fc8614ba640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc85c101170 0x7fc85c1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc85bfff640 1 -- 172.21.15.102:0/1843047714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc85c101170 msgr2=0x7fc85c1a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc85bfff640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc85c101170 0x7fc85c1a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc85bfff640 1 -- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ffa50 msgr2=0x7fc85c19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc85bfff640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ffa50 0x7fc85c19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc85bfff640 1 -- 172.21.15.102:0/1843047714 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c1a5bf0 con 0x7fc85c1007b0 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc860cb9640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ffa50 0x7fc85c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc85bfff640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c1007b0 0x7fc85c19e5b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc848009230 tx=0x7fc848031a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc859ffb640 1 -- 172.21.15.102:0/1843047714 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84803f030 con 0x7fc85c1007b0 2024-04-24T23:10:23.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc859ffb640 1 -- 172.21.15.102:0/1843047714 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc848003aa0 con 0x7fc85c1007b0 2024-04-24T23:10:23.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85c1a5e80 con 0x7fc85c1007b0 2024-04-24T23:10:23.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc859ffb640 1 -- 172.21.15.102:0/1843047714 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848035070 con 0x7fc85c1007b0 2024-04-24T23:10:23.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.969+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85c1a6280 con 0x7fc85c1007b0 2024-04-24T23:10:23.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.970+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85c1a64b0 con 0x7fc85c1007b0 2024-04-24T23:10:23.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.971+0000 7fc859ffb640 1 -- 172.21.15.102:0/1843047714 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc848042030 con 0x7fc85c1007b0 2024-04-24T23:10:23.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.972+0000 7fc859ffb640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc84c0a2b90 0x7fc84c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.972+0000 7fc860cb9640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc84c0a2b90 0x7fc84c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.972+0000 7fc859ffb640 1 -- 172.21.15.102:0/1843047714 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc8480ef0d0 con 0x7fc85c1007b0 2024-04-24T23:10:23.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.972+0000 7fc860cb9640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc84c0a2b90 0x7fc84c0a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc850005fd0 tx=0x7fc85000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:23.975+0000 7fc859ffb640 1 -- 172.21.15.102:0/1843047714 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc8480bbde0 con 0x7fc85c1007b0 2024-04-24T23:10:24.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.189+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc85c1a67f0 con 0x7fc85c1007b0 2024-04-24T23:10:24.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.190+0000 7fc859ffb640 1 -- 172.21.15.102:0/1843047714 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fc8480c00e0 con 0x7fc85c1007b0 2024-04-24T23:10:24.192 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:24.192 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:10:24.192 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 2 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.192+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc84c0a2b90 msgr2=0x7fc84c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.192+0000 7fc862f1d640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc84c0a2b90 0x7fc84c0a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc850005fd0 tx=0x7fc85000b480 comp rx=0 tx=0).stop 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.192+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c1007b0 msgr2=0x7fc85c19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.192+0000 7fc862f1d640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c1007b0 0x7fc85c19e5b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc848009230 tx=0x7fc848031a30 comp rx=0 tx=0).stop 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.193+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 shutdown_connections 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.193+0000 7fc862f1d640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc84c0a2b90 0x7fc84c0a5050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.193+0000 7fc862f1d640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc85c101170 0x7fc85c1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.193+0000 7fc862f1d640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c1007b0 0x7fc85c19e5b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.193+0000 7fc862f1d640 1 --2- 172.21.15.102:0/1843047714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ffa50 0x7fc85c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.193+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 >> 172.21.15.102:0/1843047714 conn(0x7fc85c0fb380 msgr2=0x7fc85c0fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.193+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 shutdown_connections 2024-04-24T23:10:24.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:24.193+0000 7fc862f1d640 1 -- 172.21.15.102:0/1843047714 wait complete. 2024-04-24T23:10:24.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.183:6832/2514459470,v1:172.21.15.183:6833/2514459470] up:boot 2024-04-24T23:10:24.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: fsmap 1 up:standby 2024-04-24T23:10:24.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:10:24.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:24.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:24.495 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:24.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:24.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:24.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:24.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: Deploying daemon mds.k on smithi161 2024-04-24T23:10:24.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1843047714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:24.496 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:24 smithi161 ceph-mon[32709]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.183:6832/2514459470,v1:172.21.15.183:6833/2514459470] up:boot 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: fsmap 1 up:standby 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:24.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: Deploying daemon mds.k on smithi161 2024-04-24T23:10:24.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1843047714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:24.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:24 smithi102 ceph-mon[29216]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:24.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.183:6832/2514459470,v1:172.21.15.183:6833/2514459470] up:boot 2024-04-24T23:10:24.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: fsmap 1 up:standby 2024-04-24T23:10:24.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:10:24.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:24.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:24.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:24.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:24.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:24.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:24.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: Deploying daemon mds.k on smithi161 2024-04-24T23:10:24.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1843047714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:24.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:24 smithi183 ceph-mon[33137]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:25.860 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:10:25 smithi161 systemd[1]: Starting Ceph mds.k for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:26.199 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:10:25 smithi161 podman[62058]: 2024-04-24 23:10:25.828990568 +0000 UTC m=+0.017948974 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:26.199 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:10:25 smithi161 podman[62058]: 2024-04-24 23:10:25.945547594 +0000 UTC m=+0.134506001 container create 15bbc090efde0afe2d53f863cb88884cf22617cf6a599caf240e1cc618c20436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-k, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:10:26.199 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:10:26 smithi161 podman[62058]: 2024-04-24 23:10:26.1984498 +0000 UTC m=+0.387408211 container init 15bbc090efde0afe2d53f863cb88884cf22617cf6a599caf240e1cc618c20436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:10:26.206 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 -- 172.21.15.102:0/3855941284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 msgr2=0x7f8894101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 --2- 172.21.15.102:0/3855941284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 0x7f8894101bb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8888009900 tx=0x7f888802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 -- 172.21.15.102:0/3855941284 shutdown_connections 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 --2- 172.21.15.102:0/3855941284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8894103340 0x7f8894109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 --2- 172.21.15.102:0/3855941284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8894102980 0x7f8894102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 --2- 172.21.15.102:0/3855941284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 0x7f8894101bb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 -- 172.21.15.102:0/3855941284 >> 172.21.15.102:0/3855941284 conn(0x7f88940fcd20 msgr2=0x7f88940ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 -- 172.21.15.102:0/3855941284 shutdown_connections 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 -- 172.21.15.102:0/3855941284 wait complete. 2024-04-24T23:10:26.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.320+0000 7f889b215640 1 Processor -- start 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f889b215640 1 -- start start 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f889b215640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 0x7f88941979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f889b215640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8894102980 0x7f8894197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f889b215640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8894103340 0x7f889419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f889b215640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f889410d290 con 0x7f8894101790 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f889b215640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f889410d110 con 0x7f8894103340 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f889b215640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f889410d410 con 0x7f8894102980 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8898fb1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 0x7f88941979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8898fb1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 0x7f88941979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59940/0 (socket says 172.21.15.102:59940) 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8898fb1640 1 -- 172.21.15.102:0/4140069923 learned_addr learned my addr 172.21.15.102:0/4140069923 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f88997b2640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8894103340 0x7f889419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8898fb1640 1 -- 172.21.15.102:0/4140069923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8894102980 msgr2=0x7f8894197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8898fb1640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8894102980 0x7f8894197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8898fb1640 1 -- 172.21.15.102:0/4140069923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8894103340 msgr2=0x7f889419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8898fb1640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8894103340 0x7f889419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8898fb1640 1 -- 172.21.15.102:0/4140069923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889419f5e0 con 0x7f8894101790 2024-04-24T23:10:26.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8898fb1640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 0x7f88941979c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88880314e0 tx=0x7f88880042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:26.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8891ffb640 1 -- 172.21.15.102:0/4140069923 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888803f030 con 0x7f8894101790 2024-04-24T23:10:26.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8891ffb640 1 -- 172.21.15.102:0/4140069923 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8888003cb0 con 0x7f8894101790 2024-04-24T23:10:26.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f8891ffb640 1 -- 172.21.15.102:0/4140069923 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888035070 con 0x7f8894101790 2024-04-24T23:10:26.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.321+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889419f870 con 0x7f8894101790 2024-04-24T23:10:26.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.322+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889419fdb0 con 0x7f8894101790 2024-04-24T23:10:26.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.323+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8894108b80 con 0x7f8894101790 2024-04-24T23:10:26.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.324+0000 7f8891ffb640 1 -- 172.21.15.102:0/4140069923 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8888042030 con 0x7f8894101790 2024-04-24T23:10:26.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.324+0000 7f8891ffb640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f88840a2c10 0x7f88840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:26.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.324+0000 7f8893fff640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f88840a2c10 0x7f88840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:26.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.324+0000 7f8891ffb640 1 -- 172.21.15.102:0/4140069923 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f88880ef130 con 0x7f8894101790 2024-04-24T23:10:26.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.325+0000 7f8893fff640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f88840a2c10 0x7f88840a50d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8894198ed0 tx=0x7f8880005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:26.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.329+0000 7f8891ffb640 1 -- 172.21.15.102:0/4140069923 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f88880bc530 con 0x7f8894101790 2024-04-24T23:10:26.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.561+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f889402d400 con 0x7f8894101790 2024-04-24T23:10:26.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.561+0000 7f8891ffb640 1 -- 172.21.15.102:0/4140069923 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f88880bbe30 con 0x7f8894101790 2024-04-24T23:10:26.563 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:26.563 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:10:26.564 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 3 2024-04-24T23:10:26.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f88840a2c10 msgr2=0x7f88840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f88840a2c10 0x7f88840a50d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8894198ed0 tx=0x7f8880005bf0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 msgr2=0x7f88941979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 0x7f88941979c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88880314e0 tx=0x7f88880042e0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 shutdown_connections 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f88840a2c10 0x7f88840a50d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8894103340 0x7f889419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8894102980 0x7f8894197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 --2- 172.21.15.102:0/4140069923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8894101790 0x7f88941979c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.563+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 >> 172.21.15.102:0/4140069923 conn(0x7f88940fcd20 msgr2=0x7f8894105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.564+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 shutdown_connections 2024-04-24T23:10:26.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:26.564+0000 7f889b215640 1 -- 172.21.15.102:0/4140069923 wait complete. 2024-04-24T23:10:26.609 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:10:26 smithi161 podman[62058]: 2024-04-24 23:10:26.200971778 +0000 UTC m=+0.389930187 container start 15bbc090efde0afe2d53f863cb88884cf22617cf6a599caf240e1cc618c20436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T23:10:26.609 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:10:26 smithi161 ceph-mds[62078]: -- 172.21.15.161:0/996759591 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556701391a40 con 0x556701364c00 2024-04-24T23:10:26.609 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:10:26 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-k[62074]: starting mds.k at 2024-04-24T23:10:26.609 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:10:26 smithi161 bash[62058]: 15bbc090efde0afe2d53f863cb88884cf22617cf6a599caf240e1cc618c20436 2024-04-24T23:10:26.609 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:10:26 smithi161 systemd[1]: Started Ceph mds.k for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.161:6832/756969355,v1:172.21.15.161:6833/756969355] up:boot 2024-04-24T23:10:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: fsmap 2 up:standby 2024-04-24T23:10:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:10:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:27.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:27.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:27.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:27.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:27.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:27 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4140069923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.161:6832/756969355,v1:172.21.15.161:6833/756969355] up:boot 2024-04-24T23:10:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: fsmap 2 up:standby 2024-04-24T23:10:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:10:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:27 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4140069923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:27.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:27.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.161:6832/756969355,v1:172.21.15.161:6833/756969355] up:boot 2024-04-24T23:10:27.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: fsmap 2 up:standby 2024-04-24T23:10:27.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:10:27.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:27.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:27.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:27.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:27.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:27.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:27.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:27 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4140069923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:28.420 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:28 smithi102 ceph-mon[29216]: Deploying daemon mds.j on smithi102 2024-04-24T23:10:28.577 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:28 smithi161 ceph-mon[32709]: Deploying daemon mds.j on smithi102 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 -- 172.21.15.102:0/617685055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec101890 msgr2=0x7f3eec101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/617685055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec101890 0x7f3eec101d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009900 tx=0x7f3ed402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 -- 172.21.15.102:0/617685055 shutdown_connections 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/617685055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec102250 0x7f3eec108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/617685055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec101890 0x7f3eec101d10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/617685055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec1006a0 0x7f3eec100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 -- 172.21.15.102:0/617685055 >> 172.21.15.102:0/617685055 conn(0x7f3eec0fbc30 msgr2=0x7f3eec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 -- 172.21.15.102:0/617685055 shutdown_connections 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 -- 172.21.15.102:0/617685055 wait complete. 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.684+0000 7f3ef12ee640 1 Processor -- start 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ef12ee640 1 -- start start 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ef12ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec1006a0 0x7f3eec197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:28.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ef12ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec101890 0x7f3eec197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ef12ee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec102250 0x7f3eec19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ef12ee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10c220 con 0x7f3eec102250 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ef12ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10c0a0 con 0x7f3eec101890 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ef12ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10c3a0 con 0x7f3eec1006a0 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec102250 0x7f3eec19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec102250 0x7f3eec19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43748/0 (socket says 172.21.15.102:43748) 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeb7fe640 1 -- 172.21.15.102:0/3049120359 learned_addr learned my addr 172.21.15.102:0/3049120359 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ee3fff640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec101890 0x7f3eec197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeaffd640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec1006a0 0x7f3eec197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeb7fe640 1 -- 172.21.15.102:0/3049120359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec1006a0 msgr2=0x7f3eec197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeb7fe640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec1006a0 0x7f3eec197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeb7fe640 1 -- 172.21.15.102:0/3049120359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec101890 msgr2=0x7f3eec197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeb7fe640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec101890 0x7f3eec197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:28.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeb7fe640 1 -- 172.21.15.102:0/3049120359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec19f670 con 0x7f3eec102250 2024-04-24T23:10:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ee3fff640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec101890 0x7f3eec197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3eeb7fe640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec102250 0x7f3eec19f000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400eaa0 tx=0x7f3ee400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.685+0000 7f3ee8ff9640 1 -- 172.21.15.102:0/3049120359 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400cdb0 con 0x7f3eec102250 2024-04-24T23:10:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.686+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec19f960 con 0x7f3eec102250 2024-04-24T23:10:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.686+0000 7f3ee8ff9640 1 -- 172.21.15.102:0/3049120359 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ee4005020 con 0x7f3eec102250 2024-04-24T23:10:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.686+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec19fe80 con 0x7f3eec102250 2024-04-24T23:10:28.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.686+0000 7f3ee8ff9640 1 -- 172.21.15.102:0/3049120359 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40116d0 con 0x7f3eec102250 2024-04-24T23:10:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.687+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec067cd0 con 0x7f3eec102250 2024-04-24T23:10:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.688+0000 7f3ee8ff9640 1 -- 172.21.15.102:0/3049120359 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3ee40051d0 con 0x7f3eec102250 2024-04-24T23:10:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.688+0000 7f3ee8ff9640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ed80a2b90 0x7f3ed80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.689+0000 7f3eeaffd640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ed80a2b90 0x7f3ed80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.689+0000 7f3ee8ff9640 1 -- 172.21.15.102:0/3049120359 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3ee40cedc0 con 0x7f3eec102250 2024-04-24T23:10:28.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.689+0000 7f3eeaffd640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ed80a2b90 0x7f3ed80a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3edc005fd0 tx=0x7f3edc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:28.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.695+0000 7f3ee8ff9640 1 -- 172.21.15.102:0/3049120359 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ee4016030 con 0x7f3eec102250 2024-04-24T23:10:28.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:28 smithi183 ceph-mon[33137]: Deploying daemon mds.j on smithi102 2024-04-24T23:10:28.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.917+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3eec1a0300 con 0x7f3eec102250 2024-04-24T23:10:28.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.917+0000 7f3ee8ff9640 1 -- 172.21.15.102:0/3049120359 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f3ee409c1c0 con 0x7f3eec102250 2024-04-24T23:10:28.919 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:28.920 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:10:28.920 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 3 2024-04-24T23:10:28.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ed80a2b90 msgr2=0x7f3ed80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:28.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ed80a2b90 0x7f3ed80a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3edc005fd0 tx=0x7f3edc00b390 comp rx=0 tx=0).stop 2024-04-24T23:10:28.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec102250 msgr2=0x7f3eec19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:28.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec102250 0x7f3eec19f000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400eaa0 tx=0x7f3ee400ef60 comp rx=0 tx=0).stop 2024-04-24T23:10:28.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 shutdown_connections 2024-04-24T23:10:28.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ed80a2b90 0x7f3ed80a5050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:28.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3eec102250 0x7f3eec19f000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:28.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec101890 0x7f3eec197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:28.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 --2- 172.21.15.102:0/3049120359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec1006a0 0x7f3eec197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:28.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 >> 172.21.15.102:0/3049120359 conn(0x7f3eec0fbc30 msgr2=0x7f3eec069b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:28.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 shutdown_connections 2024-04-24T23:10:28.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:28.920+0000 7f3ef12ee640 1 -- 172.21.15.102:0/3049120359 wait complete. 2024-04-24T23:10:29.072 INFO:journalctl@ceph.mds.j.smithi102.stdout:Apr 24 23:10:29 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-j[69993]: starting mds.j at 2024-04-24T23:10:29.571 INFO:journalctl@ceph.mds.j.smithi102.stdout:Apr 24 23:10:29 smithi102 bash[69975]: 4ae03cbd359f9d19018af8ec9128126154c77c6628fb42dd3f5ae48ba6650f89 2024-04-24T23:10:29.572 INFO:journalctl@ceph.mds.j.smithi102.stdout:Apr 24 23:10:29 smithi102 systemd[1]: Started Ceph mds.j for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:29 smithi102 ceph-mon[29216]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:29 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3049120359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:29.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:29.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:29.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:29 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:29 smithi161 ceph-mon[32709]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:29 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3049120359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:29.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:29.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:29.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:29.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:29 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:29 smithi183 ceph-mon[33137]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:29 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3049120359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:29.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:29 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:30.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:30 smithi102 ceph-mon[29216]: Deploying daemon mds.d on smithi102 2024-04-24T23:10:30.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:30 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.102:6834/1487977595,v1:172.21.15.102:6835/1487977595] up:boot 2024-04-24T23:10:30.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:30 smithi102 ceph-mon[29216]: fsmap 3 up:standby 2024-04-24T23:10:30.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:30 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:10:30.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:30 smithi161 ceph-mon[32709]: Deploying daemon mds.d on smithi102 2024-04-24T23:10:30.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:30 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.102:6834/1487977595,v1:172.21.15.102:6835/1487977595] up:boot 2024-04-24T23:10:30.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:30 smithi161 ceph-mon[32709]: fsmap 3 up:standby 2024-04-24T23:10:30.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:30 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:10:30.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:30 smithi183 ceph-mon[33137]: Deploying daemon mds.d on smithi102 2024-04-24T23:10:30.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:30 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.102:6834/1487977595,v1:172.21.15.102:6835/1487977595] up:boot 2024-04-24T23:10:30.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:30 smithi183 ceph-mon[33137]: fsmap 3 up:standby 2024-04-24T23:10:30.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:30 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:10:30.935 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:31.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.050+0000 7f06d6e53640 1 -- 172.21.15.102:0/1995854572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 msgr2=0x7f06d0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:31.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.050+0000 7f06d6e53640 1 --2- 172.21.15.102:0/1995854572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 0x7f06d0101cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009ca0 tx=0x7f06c002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.050+0000 7f06d6e53640 1 -- 172.21.15.102:0/1995854572 shutdown_connections 2024-04-24T23:10:31.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.050+0000 7f06d6e53640 1 --2- 172.21.15.102:0/1995854572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0102230 0x7f06d0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.050+0000 7f06d6e53640 1 --2- 172.21.15.102:0/1995854572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 0x7f06d0101cf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.050+0000 7f06d6e53640 1 --2- 172.21.15.102:0/1995854572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d0100680 0x7f06d0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.050+0000 7f06d6e53640 1 -- 172.21.15.102:0/1995854572 >> 172.21.15.102:0/1995854572 conn(0x7f06d00fbbf0 msgr2=0x7f06d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.050+0000 7f06d6e53640 1 -- 172.21.15.102:0/1995854572 shutdown_connections 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.050+0000 7f06d6e53640 1 -- 172.21.15.102:0/1995854572 wait complete. 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d6e53640 1 Processor -- start 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d6e53640 1 -- start start 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d6e53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d0100680 0x7f06d0197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d6e53640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 0x7f06d0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d6e53640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0102230 0x7f06d019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d6e53640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06d010c1b0 con 0x7f06d0101870 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d6e53640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f06d010c030 con 0x7f06d0102230 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d6e53640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06d010c330 con 0x7f06d0100680 2024-04-24T23:10:31.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06cffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 0x7f06d0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06cffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 0x7f06d0197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43774/0 (socket says 172.21.15.102:43774) 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06cffff640 1 -- 172.21.15.102:0/804097178 learned_addr learned my addr 172.21.15.102:0/804097178 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d53f0640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0102230 0x7f06d019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06d4bef640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d0100680 0x7f06d0197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06cffff640 1 -- 172.21.15.102:0/804097178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d0100680 msgr2=0x7f06d0197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.051+0000 7f06cffff640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d0100680 0x7f06d0197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.052+0000 7f06cffff640 1 -- 172.21.15.102:0/804097178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0102230 msgr2=0x7f06d019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.052+0000 7f06cffff640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0102230 0x7f06d019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.052+0000 7f06cffff640 1 -- 172.21.15.102:0/804097178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d019f620 con 0x7f06d0101870 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.052+0000 7f06cffff640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 0x7f06d0197f40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009dc0 tx=0x7f06c0009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.052+0000 7f06cdffb640 1 -- 172.21.15.102:0/804097178 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0003db0 con 0x7f06d0101870 2024-04-24T23:10:31.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.052+0000 7f06cdffb640 1 -- 172.21.15.102:0/804097178 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06c00044f0 con 0x7f06d0101870 2024-04-24T23:10:31.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.052+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d019f8b0 con 0x7f06d0101870 2024-04-24T23:10:31.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.052+0000 7f06cdffb640 1 -- 172.21.15.102:0/804097178 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0030d90 con 0x7f06d0101870 2024-04-24T23:10:31.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.052+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d019fe00 con 0x7f06d0101870 2024-04-24T23:10:31.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.053+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d01a0030 con 0x7f06d0101870 2024-04-24T23:10:31.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.054+0000 7f06cdffb640 1 -- 172.21.15.102:0/804097178 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f06c00305c0 con 0x7f06d0101870 2024-04-24T23:10:31.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.055+0000 7f06cdffb640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f06bc0a2b90 0x7f06bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:31.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.055+0000 7f06d4bef640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f06bc0a2b90 0x7f06bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:31.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.055+0000 7f06cdffb640 1 -- 172.21.15.102:0/804097178 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f06c00effc0 con 0x7f06d0101870 2024-04-24T23:10:31.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.055+0000 7f06d4bef640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f06bc0a2b90 0x7f06bc0a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06b8005fd0 tx=0x7f06b800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:31.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.060+0000 7f06cdffb640 1 -- 172.21.15.102:0/804097178 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f06c00bd3c0 con 0x7f06d0101870 2024-04-24T23:10:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.293+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06d01a0200 con 0x7f06d0101870 2024-04-24T23:10:31.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.294+0000 7f06cdffb640 1 -- 172.21.15.102:0/804097178 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f06c00bccc0 con 0x7f06d0101870 2024-04-24T23:10:31.296 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:31.296 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:10:31.297 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 4 2024-04-24T23:10:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f06bc0a2b90 msgr2=0x7f06bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f06bc0a2b90 0x7f06bc0a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06b8005fd0 tx=0x7f06b800b390 comp rx=0 tx=0).stop 2024-04-24T23:10:31.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 msgr2=0x7f06d0197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 0x7f06d0197f40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009dc0 tx=0x7f06c0009380 comp rx=0 tx=0).stop 2024-04-24T23:10:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 shutdown_connections 2024-04-24T23:10:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f06bc0a2b90 0x7f06bc0a5050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0102230 0x7f06d019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0101870 0x7f06d0197f40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 --2- 172.21.15.102:0/804097178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06d0100680 0x7f06d0197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 >> 172.21.15.102:0/804097178 conn(0x7f06d00fbbf0 msgr2=0x7f06d0068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 shutdown_connections 2024-04-24T23:10:31.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:31.296+0000 7f06d6e53640 1 -- 172.21.15.102:0/804097178 wait complete. 2024-04-24T23:10:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:31 smithi102 ceph-mon[29216]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:31 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/804097178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:31.572 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:10:31 smithi102 systemd[1]: Starting Ceph mds.d for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:31 smithi161 ceph-mon[32709]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:31 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/804097178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:31.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:31 smithi183 ceph-mon[33137]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:31.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:31 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/804097178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:31.997 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:10:31 smithi102 podman[70329]: 2024-04-24 23:10:31.62127996 +0000 UTC m=+0.017991486 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:31.997 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:10:31 smithi102 podman[70329]: 2024-04-24 23:10:31.752638531 +0000 UTC m=+0.149350052 container create 2dcddcfd0cb40f24ee5f17b081c559dc881138bf3344ac70ff02fb807a2f8d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T23:10:31.997 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:10:31 smithi102 podman[70329]: 2024-04-24 23:10:31.996777349 +0000 UTC m=+0.393488862 container init 2dcddcfd0cb40f24ee5f17b081c559dc881138bf3344ac70ff02fb807a2f8d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:10:32.322 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:10:31 smithi102 podman[70329]: 2024-04-24 23:10:31.999389853 +0000 UTC m=+0.396101380 container start 2dcddcfd0cb40f24ee5f17b081c559dc881138bf3344ac70ff02fb807a2f8d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T23:10:32.322 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:10:32 smithi102 ceph-mds[70348]: -- 172.21.15.102:0/3681002317 <== mon.0 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559383f9a40 con 0x5559383cd000 2024-04-24T23:10:32.322 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:10:32 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d[70344]: starting mds.d at 2024-04-24T23:10:32.322 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:10:32 smithi102 bash[70329]: 2dcddcfd0cb40f24ee5f17b081c559dc881138bf3344ac70ff02fb807a2f8d9f 2024-04-24T23:10:32.322 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:10:32 smithi102 systemd[1]: Started Ceph mds.d for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:33.312 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:33.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.412+0000 7f865ddce640 1 -- 172.21.15.102:0/1000201046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 msgr2=0x7f8658108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.412+0000 7f865ddce640 1 --2- 172.21.15.102:0/1000201046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 0x7f8658108ac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8650009ca0 tx=0x7f865002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.412+0000 7f865ddce640 1 -- 172.21.15.102:0/1000201046 shutdown_connections 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.412+0000 7f865ddce640 1 --2- 172.21.15.102:0/1000201046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 0x7f8658108ac0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.412+0000 7f865ddce640 1 --2- 172.21.15.102:0/1000201046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8658101870 0x7f8658101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.412+0000 7f865ddce640 1 --2- 172.21.15.102:0/1000201046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8658100680 0x7f8658100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.412+0000 7f865ddce640 1 -- 172.21.15.102:0/1000201046 >> 172.21.15.102:0/1000201046 conn(0x7f86580fbbf0 msgr2=0x7f86580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.412+0000 7f865ddce640 1 -- 172.21.15.102:0/1000201046 shutdown_connections 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.412+0000 7f865ddce640 1 -- 172.21.15.102:0/1000201046 wait complete. 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f865ddce640 1 Processor -- start 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f865ddce640 1 -- start start 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f865ddce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8658100680 0x7f8658104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f865ddce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8658101870 0x7f8658102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f865ddce640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 0x7f8658103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f865ddce640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f865810c1b0 con 0x7f8658102230 2024-04-24T23:10:33.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f865ddce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f865810c030 con 0x7f8658100680 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f865ddce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f865810c330 con 0x7f8658101870 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f8657fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 0x7f8658103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f8657fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 0x7f8658103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43796/0 (socket says 172.21.15.102:43796) 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f8656ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8658101870 0x7f8658102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.413+0000 7f8657fff640 1 -- 172.21.15.102:0/1717493133 learned_addr learned my addr 172.21.15.102:0/1717493133 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f86577fe640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8658100680 0x7f8658104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8656ffd640 1 -- 172.21.15.102:0/1717493133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8658100680 msgr2=0x7f8658104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8656ffd640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8658100680 0x7f8658104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8656ffd640 1 -- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 msgr2=0x7f8658103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8656ffd640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 0x7f8658103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8656ffd640 1 -- 172.21.15.102:0/1717493133 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8658103a20 con 0x7f8658101870 2024-04-24T23:10:33.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8657fff640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 0x7f8658103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:10:33.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f86577fe640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8658100680 0x7f8658104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:33.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8656ffd640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8658101870 0x7f8658102c50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f864c00e9f0 tx=0x7f864c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8654ff9640 1 -- 172.21.15.102:0/1717493133 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c00ccc0 con 0x7f8658101870 2024-04-24T23:10:33.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86581a1a30 con 0x7f8658101870 2024-04-24T23:10:33.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8654ff9640 1 -- 172.21.15.102:0/1717493133 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f864c005020 con 0x7f8658101870 2024-04-24T23:10:33.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f8654ff9640 1 -- 172.21.15.102:0/1717493133 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c004a00 con 0x7f8658101870 2024-04-24T23:10:33.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.414+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86581a1f30 con 0x7f8658101870 2024-04-24T23:10:33.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.415+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86581a21c0 con 0x7f8658101870 2024-04-24T23:10:33.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.416+0000 7f8654ff9640 1 -- 172.21.15.102:0/1717493133 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f864c004ba0 con 0x7f8658101870 2024-04-24T23:10:33.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.417+0000 7f8654ff9640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f86400a2b90 0x7f86400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.417+0000 7f86577fe640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f86400a2b90 0x7f86400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.417+0000 7f8654ff9640 1 -- 172.21.15.102:0/1717493133 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f864c0cec20 con 0x7f8658101870 2024-04-24T23:10:33.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.417+0000 7f86577fe640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f86400a2b90 0x7f86400a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f86581016d0 tx=0x7f864800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.421+0000 7f8654ff9640 1 -- 172.21.15.102:0/1717493133 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f864c09bf80 con 0x7f8658101870 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: Deploying daemon mds.e on smithi161 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.102:6836/1957522131,v1:172.21.15.102:6837/1957522131] up:boot 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: fsmap 4 up:standby 2024-04-24T23:10:33.429 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:33 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:10:33.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:33.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:33.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:33.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:33.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:33.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:33.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: Deploying daemon mds.e on smithi161 2024-04-24T23:10:33.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:33.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.102:6836/1957522131,v1:172.21.15.102:6837/1957522131] up:boot 2024-04-24T23:10:33.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: fsmap 4 up:standby 2024-04-24T23:10:33.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:33 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:10:33.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:33.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:33.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:33.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:33.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:33.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:33.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: Deploying daemon mds.e on smithi161 2024-04-24T23:10:33.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:33.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.102:6836/1957522131,v1:172.21.15.102:6837/1957522131] up:boot 2024-04-24T23:10:33.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: fsmap 4 up:standby 2024-04-24T23:10:33.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:33 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:10:33.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.645+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86581a2390 con 0x7f8658101870 2024-04-24T23:10:33.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.646+0000 7f8654ff9640 1 -- 172.21.15.102:0/1717493133 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7f864c09b880 con 0x7f8658101870 2024-04-24T23:10:33.648 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:33.648 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:10:33.648 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-04-24T23:10:33.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f86400a2b90 msgr2=0x7f86400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f86400a2b90 0x7f86400a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f86581016d0 tx=0x7f864800a5e0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8658101870 msgr2=0x7f8658102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8658101870 0x7f8658102c50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f864c00e9f0 tx=0x7f864c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 shutdown_connections 2024-04-24T23:10:33.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f86400a2b90 0x7f86400a5050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8658102230 0x7f8658103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8658101870 0x7f8658102c50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 --2- 172.21.15.102:0/1717493133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8658100680 0x7f8658104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 >> 172.21.15.102:0/1717493133 conn(0x7f86580fbbf0 msgr2=0x7f86580fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 shutdown_connections 2024-04-24T23:10:33.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:33.648+0000 7f865ddce640 1 -- 172.21.15.102:0/1717493133 wait complete. 2024-04-24T23:10:34.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:34 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1717493133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:34.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:34 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1717493133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:34 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1717493133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:34.844 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:10:34 smithi161 podman[62375]: 2024-04-24 23:10:34.842797911 +0000 UTC m=+0.156515480 container create 623b1dcc038311cb77ddbbb32d22d8d829cc3abc23a9c3367f782486d4ccc6f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T23:10:35.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:35 smithi161 ceph-mon[32709]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:35.359 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:10:35 smithi161 podman[62375]: 2024-04-24 23:10:35.104268514 +0000 UTC m=+0.417986091 container init 623b1dcc038311cb77ddbbb32d22d8d829cc3abc23a9c3367f782486d4ccc6f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-24T23:10:35.359 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:10:35 smithi161 podman[62375]: 2024-04-24 23:10:35.106873808 +0000 UTC m=+0.420591381 container start 623b1dcc038311cb77ddbbb32d22d8d829cc3abc23a9c3367f782486d4ccc6f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:10:35.360 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:10:35 smithi161 ceph-mds[62394]: -- 172.21.15.161:0/3704870637 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a26be7a40 con 0x564a26bba800 2024-04-24T23:10:35.360 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:10:35 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e[62390]: starting mds.e at 2024-04-24T23:10:35.360 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:10:35 smithi161 bash[62375]: 623b1dcc038311cb77ddbbb32d22d8d829cc3abc23a9c3367f782486d4ccc6f3 2024-04-24T23:10:35.360 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:10:35 smithi161 systemd[1]: Started Ceph mds.e for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:35.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:35 smithi183 ceph-mon[33137]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:35.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:35 smithi102 ceph-mon[29216]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:35.661 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:35.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.767+0000 7f590dcda640 1 -- 172.21.15.102:0/2865542860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908100010 msgr2=0x7f5908102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.767+0000 7f590dcda640 1 --2- 172.21.15.102:0/2865542860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908100010 0x7f5908102440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009900 tx=0x7f58f802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:35.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.767+0000 7f590dcda640 1 -- 172.21.15.102:0/2865542860 shutdown_connections 2024-04-24T23:10:35.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.767+0000 7f590dcda640 1 --2- 172.21.15.102:0/2865542860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5908105330 0x7f59081057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.767+0000 7f590dcda640 1 --2- 172.21.15.102:0/2865542860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908102980 0x7f5908104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.767+0000 7f590dcda640 1 --2- 172.21.15.102:0/2865542860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908100010 0x7f5908102440 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.767+0000 7f590dcda640 1 -- 172.21.15.102:0/2865542860 >> 172.21.15.102:0/2865542860 conn(0x7f59080fbc30 msgr2=0x7f59080fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.767+0000 7f590dcda640 1 -- 172.21.15.102:0/2865542860 shutdown_connections 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.767+0000 7f590dcda640 1 -- 172.21.15.102:0/2865542860 wait complete. 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f590dcda640 1 Processor -- start 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f590dcda640 1 -- start start 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f590dcda640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908100010 0x7f590819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f590dcda640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908102980 0x7f590819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f590dcda640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5908105330 0x7f59081a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f590dcda640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f59081138f0 con 0x7f5908102980 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f590dcda640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5908113770 con 0x7f5908105330 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f590dcda640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5908113a70 con 0x7f5908100010 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f5906ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908102980 0x7f590819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f5906ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908102980 0x7f590819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43818/0 (socket says 172.21.15.102:43818) 2024-04-24T23:10:35.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f59077fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908100010 0x7f590819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f5906ffd640 1 -- 172.21.15.102:0/1561566425 learned_addr learned my addr 172.21.15.102:0/1561566425 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.768+0000 7f5907fff640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5908105330 0x7f59081a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f5906ffd640 1 -- 172.21.15.102:0/1561566425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908100010 msgr2=0x7f590819df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f5906ffd640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908100010 0x7f590819df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f5906ffd640 1 -- 172.21.15.102:0/1561566425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5908105330 msgr2=0x7f59081a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f5906ffd640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5908105330 0x7f59081a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f5906ffd640 1 -- 172.21.15.102:0/1561566425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59081a5b40 con 0x7f5908102980 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f5906ffd640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908102980 0x7f590819e460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00c920 tx=0x7f58fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f5904ff9640 1 -- 172.21.15.102:0/1561566425 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc007c70 con 0x7f5908102980 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59081a5e30 con 0x7f5908102980 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f5904ff9640 1 -- 172.21.15.102:0/1561566425 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58fc007e10 con 0x7f5908102980 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f5904ff9640 1 -- 172.21.15.102:0/1561566425 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc018820 con 0x7f5908102980 2024-04-24T23:10:35.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.769+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59081a62f0 con 0x7f5908102980 2024-04-24T23:10:35.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.770+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59080fd720 con 0x7f5908102980 2024-04-24T23:10:35.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.771+0000 7f5904ff9640 1 -- 172.21.15.102:0/1561566425 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f58fc0189c0 con 0x7f5908102980 2024-04-24T23:10:35.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.771+0000 7f5904ff9640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58f00a2b90 0x7f58f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.772+0000 7f59077fe640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58f00a2b90 0x7f58f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.772+0000 7f5904ff9640 1 -- 172.21.15.102:0/1561566425 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f58fc0cec20 con 0x7f5908102980 2024-04-24T23:10:35.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.772+0000 7f59077fe640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58f00a2b90 0x7f58f00a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f58f80314e0 tx=0x7f58f8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:35.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.776+0000 7f5904ff9640 1 -- 172.21.15.102:0/1561566425 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f58fc014030 con 0x7f5908102980 2024-04-24T23:10:35.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.989+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f590802d340 con 0x7f5908102980 2024-04-24T23:10:35.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.990+0000 7f5904ff9640 1 -- 172.21.15.102:0/1561566425 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f58fc09c020 con 0x7f5908102980 2024-04-24T23:10:35.992 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:35.992 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T23:10:35.992 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 6 2024-04-24T23:10:35.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58f00a2b90 msgr2=0x7f58f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58f00a2b90 0x7f58f00a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f58f80314e0 tx=0x7f58f8005a20 comp rx=0 tx=0).stop 2024-04-24T23:10:35.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908102980 msgr2=0x7f590819e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908102980 0x7f590819e460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00c920 tx=0x7f58fc00cde0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 shutdown_connections 2024-04-24T23:10:35.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58f00a2b90 0x7f58f00a5050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5908105330 0x7f59081a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5908102980 0x7f590819e460 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 --2- 172.21.15.102:0/1561566425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908100010 0x7f590819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 >> 172.21.15.102:0/1561566425 conn(0x7f59080fbc30 msgr2=0x7f5908103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:35.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 shutdown_connections 2024-04-24T23:10:35.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:35.992+0000 7f590dcda640 1 -- 172.21.15.102:0/1561566425 wait complete. 2024-04-24T23:10:36.189 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.161:6834/2778927873,v1:172.21.15.161:6835/2778927873] up:boot 2024-04-24T23:10:36.189 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: fsmap 5 up:standby 2024-04-24T23:10:36.189 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:10:36.189 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:36.190 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:36.190 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:36.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.161:6834/2778927873,v1:172.21.15.161:6835/2778927873] up:boot 2024-04-24T23:10:36.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: fsmap 5 up:standby 2024-04-24T23:10:36.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:10:36.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:36.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:36.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:36.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:36.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:36.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:36.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: Deploying daemon mds.f on smithi183 2024-04-24T23:10:36.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:36 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1561566425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:36.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:36.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:36.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:36.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: Deploying daemon mds.f on smithi183 2024-04-24T23:10:36.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:36 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1561566425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.161:6834/2778927873,v1:172.21.15.161:6835/2778927873] up:boot 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: fsmap 5 up:standby 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: Deploying daemon mds.f on smithi183 2024-04-24T23:10:36.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:36 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1561566425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:37.512 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:10:37 smithi183 systemd[1]: Starting Ceph mds.f for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:37.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:37 smithi183 ceph-mon[33137]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:37.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:37 smithi102 ceph-mon[29216]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:37 smithi161 ceph-mon[32709]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:37.915 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:10:37 smithi183 podman[63056]: 2024-04-24 23:10:37.523416613 +0000 UTC m=+0.017412741 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:37.915 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:10:37 smithi183 podman[63056]: 2024-04-24 23:10:37.636201038 +0000 UTC m=+0.130197166 container create 57a123290fd9f2afb88b13559de85e2ef5eaa24d00dac78e4281cd2a79e4f56b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T23:10:37.915 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:10:37 smithi183 podman[63056]: 2024-04-24 23:10:37.914341767 +0000 UTC m=+0.408337896 container init 57a123290fd9f2afb88b13559de85e2ef5eaa24d00dac78e4281cd2a79e4f56b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T23:10:38.006 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:38.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 -- 172.21.15.102:0/1321185690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930102240 msgr2=0x7f1930108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 --2- 172.21.15.102:0/1321185690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930102240 0x7f1930108ad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1928009ca0 tx=0x7f192802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 -- 172.21.15.102:0/1321185690 shutdown_connections 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 --2- 172.21.15.102:0/1321185690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930102240 0x7f1930108ad0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 --2- 172.21.15.102:0/1321185690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1930101880 0x7f1930101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 --2- 172.21.15.102:0/1321185690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930100690 0x7f1930100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 -- 172.21.15.102:0/1321185690 >> 172.21.15.102:0/1321185690 conn(0x7f19300fbc40 msgr2=0x7f19300fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 -- 172.21.15.102:0/1321185690 shutdown_connections 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 -- 172.21.15.102:0/1321185690 wait complete. 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 Processor -- start 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.114+0000 7f1938134640 1 -- start start 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1938134640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930100690 0x7f1930197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1938134640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1930101880 0x7f1930197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1938134640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102240 0x7f193019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1938134640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f193010d2e0 con 0x7f1930100690 2024-04-24T23:10:38.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1938134640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f193010d160 con 0x7f1930102240 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1938134640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f193010d460 con 0x7f1930101880 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1935ed0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930100690 0x7f1930197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1935ed0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930100690 0x7f1930197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53426/0 (socket says 172.21.15.102:53426) 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1935ed0640 1 -- 172.21.15.102:0/945794634 learned_addr learned my addr 172.21.15.102:0/945794634 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f19356cf640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1930101880 0x7f1930197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f19366d1640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102240 0x7f193019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1935ed0640 1 -- 172.21.15.102:0/945794634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1930101880 msgr2=0x7f1930197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1935ed0640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1930101880 0x7f1930197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1935ed0640 1 -- 172.21.15.102:0/945794634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102240 msgr2=0x7f193019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1935ed0640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102240 0x7f193019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1935ed0640 1 -- 172.21.15.102:0/945794634 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193019f520 con 0x7f1930100690 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f19366d1640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102240 0x7f193019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1935ed0640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930100690 0x7f1930197900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f192400c970 tx=0x7f192400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1922ffd640 1 -- 172.21.15.102:0/945794634 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924007c10 con 0x7f1930100690 2024-04-24T23:10:38.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193019f810 con 0x7f1930100690 2024-04-24T23:10:38.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1922ffd640 1 -- 172.21.15.102:0/945794634 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1924007db0 con 0x7f1930100690 2024-04-24T23:10:38.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1922ffd640 1 -- 172.21.15.102:0/945794634 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192400f7d0 con 0x7f1930100690 2024-04-24T23:10:38.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.115+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193019fdb0 con 0x7f1930100690 2024-04-24T23:10:38.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.116+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1930107950 con 0x7f1930100690 2024-04-24T23:10:38.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.117+0000 7f1922ffd640 1 -- 172.21.15.102:0/945794634 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f192400fa70 con 0x7f1930100690 2024-04-24T23:10:38.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.118+0000 7f1922ffd640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f191c0a2b90 0x7f191c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.118+0000 7f19356cf640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f191c0a2b90 0x7f191c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.118+0000 7f1922ffd640 1 -- 172.21.15.102:0/945794634 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f19240cef30 con 0x7f1930100690 2024-04-24T23:10:38.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.118+0000 7f19356cf640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f191c0a2b90 0x7f191c0a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1930198e10 tx=0x7f192c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.125+0000 7f1922ffd640 1 -- 172.21.15.102:0/945794634 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f192409c3c0 con 0x7f1930100690 2024-04-24T23:10:38.262 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:10:37 smithi183 podman[63056]: 2024-04-24 23:10:37.916984057 +0000 UTC m=+0.410980196 container start 57a123290fd9f2afb88b13559de85e2ef5eaa24d00dac78e4281cd2a79e4f56b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T23:10:38.262 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:10:37 smithi183 ceph-mds[63075]: -- 172.21.15.183:0/2646106607 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564657cd5a40 con 0x564657ca8800 2024-04-24T23:10:38.262 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:10:37 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f[63071]: starting mds.f at 2024-04-24T23:10:38.262 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:10:38 smithi183 bash[63056]: 57a123290fd9f2afb88b13559de85e2ef5eaa24d00dac78e4281cd2a79e4f56b 2024-04-24T23:10:38.262 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:10:38 smithi183 systemd[1]: Started Ceph mds.f for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:38.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.342+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f193002d380 con 0x7f1930100690 2024-04-24T23:10:38.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.343+0000 7f1922ffd640 1 -- 172.21.15.102:0/945794634 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7f192409bcc0 con 0x7f1930100690 2024-04-24T23:10:38.345 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:38.345 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T23:10:38.345 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 7 2024-04-24T23:10:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f191c0a2b90 msgr2=0x7f191c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f191c0a2b90 0x7f191c0a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1930198e10 tx=0x7f192c005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930100690 msgr2=0x7f1930197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930100690 0x7f1930197900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f192400c970 tx=0x7f192400ce30 comp rx=0 tx=0).stop 2024-04-24T23:10:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 shutdown_connections 2024-04-24T23:10:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f191c0a2b90 0x7f191c0a5050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1930102240 0x7f193019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1930101880 0x7f1930197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 --2- 172.21.15.102:0/945794634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930100690 0x7f1930197900 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 >> 172.21.15.102:0/945794634 conn(0x7f19300fbc40 msgr2=0x7f1930104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 shutdown_connections 2024-04-24T23:10:38.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:38.345+0000 7f1938134640 1 -- 172.21.15.102:0/945794634 wait complete. 2024-04-24T23:10:39.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: Deploying daemon mds.c on smithi183 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.183:6834/2266462643,v1:172.21.15.183:6835/2266462643] up:boot 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: fsmap 6 up:standby 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:39.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:39 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/945794634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: Deploying daemon mds.c on smithi183 2024-04-24T23:10:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.183:6834/2266462643,v1:172.21.15.183:6835/2266462643] up:boot 2024-04-24T23:10:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: fsmap 6 up:standby 2024-04-24T23:10:39.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:10:39.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:39.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:39 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/945794634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:39.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:39.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: Deploying daemon mds.c on smithi183 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.183:6834/2266462643,v1:172.21.15.183:6835/2266462643] up:boot 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: fsmap 6 up:standby 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:39.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:39 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/945794634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:40.358 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:40.442 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:10:40 smithi183 systemd[1]: Starting Ceph mds.c for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 -- 172.21.15.102:0/2618088746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c4102270 msgr2=0x7f58c4108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 --2- 172.21.15.102:0/2618088746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c4102270 0x7f58c4108b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009ca0 tx=0x7f58bc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 -- 172.21.15.102:0/2618088746 shutdown_connections 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 --2- 172.21.15.102:0/2618088746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c4102270 0x7f58c4108b00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 --2- 172.21.15.102:0/2618088746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58c41018b0 0x7f58c4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 --2- 172.21.15.102:0/2618088746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58c41006c0 0x7f58c4100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 -- 172.21.15.102:0/2618088746 >> 172.21.15.102:0/2618088746 conn(0x7f58c40fbc30 msgr2=0x7f58c40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 -- 172.21.15.102:0/2618088746 shutdown_connections 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 -- 172.21.15.102:0/2618088746 wait complete. 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.469+0000 7f58c9695640 1 Processor -- start 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c9695640 1 -- start start 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c9695640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58c41006c0 0x7f58c4197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c9695640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c41018b0 0x7f58c4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c9695640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58c4102270 0x7f58c419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c9695640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58c410c1d0 con 0x7f58c41018b0 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c9695640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f58c410c050 con 0x7f58c41006c0 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c9695640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58c410c350 con 0x7f58c4102270 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c27fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c41018b0 0x7f58c4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c27fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c41018b0 0x7f58c4197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53448/0 (socket says 172.21.15.102:53448) 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c27fc640 1 -- 172.21.15.102:0/262939363 learned_addr learned my addr 172.21.15.102:0/262939363 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c2ffd640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58c41006c0 0x7f58c4197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c37fe640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58c4102270 0x7f58c419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c27fc640 1 -- 172.21.15.102:0/262939363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58c4102270 msgr2=0x7f58c419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c27fc640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58c4102270 0x7f58c419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c27fc640 1 -- 172.21.15.102:0/262939363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58c41006c0 msgr2=0x7f58c4197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c27fc640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58c41006c0 0x7f58c4197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c27fc640 1 -- 172.21.15.102:0/262939363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c419f5b0 con 0x7f58c41018b0 2024-04-24T23:10:40.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c27fc640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c41018b0 0x7f58c4197f90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f58b400d950 tx=0x7f58b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58abfff640 1 -- 172.21.15.102:0/262939363 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4017070 con 0x7f58c41018b0 2024-04-24T23:10:40.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c419f8a0 con 0x7f58c41018b0 2024-04-24T23:10:40.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.470+0000 7f58abfff640 1 -- 172.21.15.102:0/262939363 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58b4009c30 con 0x7f58c41018b0 2024-04-24T23:10:40.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.471+0000 7f58abfff640 1 -- 172.21.15.102:0/262939363 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b400f620 con 0x7f58c41018b0 2024-04-24T23:10:40.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.471+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c419fdf0 con 0x7f58c41018b0 2024-04-24T23:10:40.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.472+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c419fff0 con 0x7f58c41018b0 2024-04-24T23:10:40.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.473+0000 7f58abfff640 1 -- 172.21.15.102:0/262939363 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f58b40109a0 con 0x7f58c41018b0 2024-04-24T23:10:40.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.473+0000 7f58abfff640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58ac0a2b90 0x7f58ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:40.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.473+0000 7f58c2ffd640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58ac0a2b90 0x7f58ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:40.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.473+0000 7f58abfff640 1 -- 172.21.15.102:0/262939363 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f58b40ce7f0 con 0x7f58c41018b0 2024-04-24T23:10:40.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.474+0000 7f58c2ffd640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58ac0a2b90 0x7f58ac0a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58c4101710 tx=0x7f58b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.478+0000 7f58abfff640 1 -- 172.21.15.102:0/262939363 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f58b4009de0 con 0x7f58c41018b0 2024-04-24T23:10:40.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.690+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58c41a0380 con 0x7f58c41018b0 2024-04-24T23:10:40.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.691+0000 7f58abfff640 1 -- 172.21.15.102:0/262939363 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7f58b409bbf0 con 0x7f58c41018b0 2024-04-24T23:10:40.693 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:40.693 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T23:10:40.693 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 7 2024-04-24T23:10:40.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58ac0a2b90 msgr2=0x7f58ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58ac0a2b90 0x7f58ac0a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58c4101710 tx=0x7f58b8005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:40.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c41018b0 msgr2=0x7f58c4197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:40.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c41018b0 0x7f58c4197f90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f58b400d950 tx=0x7f58b400de10 comp rx=0 tx=0).stop 2024-04-24T23:10:40.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 shutdown_connections 2024-04-24T23:10:40.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f58ac0a2b90 0x7f58ac0a5050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58c4102270 0x7f58c419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c41018b0 0x7f58c4197f90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 --2- 172.21.15.102:0/262939363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58c41006c0 0x7f58c4197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:40.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 >> 172.21.15.102:0/262939363 conn(0x7f58c40fbc30 msgr2=0x7f58c40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:40.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 shutdown_connections 2024-04-24T23:10:40.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:40.693+0000 7f58c9695640 1 -- 172.21.15.102:0/262939363 wait complete. 2024-04-24T23:10:40.711 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:10:40 smithi183 podman[63362]: 2024-04-24 23:10:40.440995155 +0000 UTC m=+0.101919682 container create f2e55b8c423ee779ede4fdd0c5f3b8713642933f4b58ed3221778f8691b73376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:10:40.711 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:10:40 smithi183 podman[63362]: 2024-04-24 23:10:40.356579122 +0000 UTC m=+0.017503654 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:40.711 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:10:40 smithi183 podman[63362]: 2024-04-24 23:10:40.710528541 +0000 UTC m=+0.371453067 container init f2e55b8c423ee779ede4fdd0c5f3b8713642933f4b58ed3221778f8691b73376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T23:10:41.012 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:10:40 smithi183 podman[63362]: 2024-04-24 23:10:40.713288989 +0000 UTC m=+0.374213516 container start f2e55b8c423ee779ede4fdd0c5f3b8713642933f4b58ed3221778f8691b73376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T23:10:41.013 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:10:40 smithi183 ceph-mds[63381]: -- 172.21.15.183:0/2234341166 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f85ae1da40 con 0x55f85adf0800 2024-04-24T23:10:41.013 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:10:40 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c[63377]: starting mds.c at 2024-04-24T23:10:41.013 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:10:40 smithi183 bash[63362]: f2e55b8c423ee779ede4fdd0c5f3b8713642933f4b58ed3221778f8691b73376 2024-04-24T23:10:41.013 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:10:40 smithi183 systemd[1]: Started Ceph mds.c for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:41.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:41 smithi161 ceph-mon[32709]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:41.552 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:41 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/262939363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:41.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:41.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:41.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:41.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:41.553 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:41 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:41.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:41 smithi102 ceph-mon[29216]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:41 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/262939363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:41 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:41.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:41 smithi183 ceph-mon[33137]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:41.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:41 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/262939363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:41.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:41.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:41.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:41.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:41.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:41 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:42.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:42 smithi102 ceph-mon[29216]: Deploying daemon mds.h on smithi161 2024-04-24T23:10:42.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:42 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.183:6836/3933349751,v1:172.21.15.183:6837/3933349751] up:boot 2024-04-24T23:10:42.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:42 smithi102 ceph-mon[29216]: fsmap 7 up:standby 2024-04-24T23:10:42.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:42 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:10:42.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:42 smithi161 ceph-mon[32709]: Deploying daemon mds.h on smithi161 2024-04-24T23:10:42.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:42 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.183:6836/3933349751,v1:172.21.15.183:6837/3933349751] up:boot 2024-04-24T23:10:42.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:42 smithi161 ceph-mon[32709]: fsmap 7 up:standby 2024-04-24T23:10:42.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:42 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:10:42.707 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:42.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:42 smithi183 ceph-mon[33137]: Deploying daemon mds.h on smithi161 2024-04-24T23:10:42.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:42 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.183:6836/3933349751,v1:172.21.15.183:6837/3933349751] up:boot 2024-04-24T23:10:42.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:42 smithi183 ceph-mon[33137]: fsmap 7 up:standby 2024-04-24T23:10:42.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:42 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 -- 172.21.15.102:0/1252933581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c8107500 msgr2=0x7f59c81098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 --2- 172.21.15.102:0/1252933581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c8107500 0x7f59c81098f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009ca0 tx=0x7f59c002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 -- 172.21.15.102:0/1252933581 shutdown_connections 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 --2- 172.21.15.102:0/1252933581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c8107500 0x7f59c81098f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 --2- 172.21.15.102:0/1252933581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59c8104bd0 0x7f59c8106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 --2- 172.21.15.102:0/1252933581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59c81022a0 0x7f59c8104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 -- 172.21.15.102:0/1252933581 >> 172.21.15.102:0/1252933581 conn(0x7f59c80fbbf0 msgr2=0x7f59c80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 -- 172.21.15.102:0/1252933581 shutdown_connections 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 -- 172.21.15.102:0/1252933581 wait complete. 2024-04-24T23:10:42.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.816+0000 7f59ce142640 1 Processor -- start 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59ce142640 1 -- start start 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59ce142640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c81022a0 0x7f59c8199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59ce142640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59c8104bd0 0x7f59c819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59ce142640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59c8107500 0x7f59c81a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59ce142640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810d3a0 con 0x7f59c81022a0 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59ce142640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810d220 con 0x7f59c8104bd0 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59ce142640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810d520 con 0x7f59c8107500 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c81022a0 0x7f59c8199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c81022a0 0x7f59c8199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53460/0 (socket says 172.21.15.102:53460) 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c77fe640 1 -- 172.21.15.102:0/2683318108 learned_addr learned my addr 172.21.15.102:0/2683318108 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c7fff640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59c8107500 0x7f59c81a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:42.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c6ffd640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59c8104bd0 0x7f59c819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c77fe640 1 -- 172.21.15.102:0/2683318108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59c8107500 msgr2=0x7f59c81a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c77fe640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59c8107500 0x7f59c81a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c77fe640 1 -- 172.21.15.102:0/2683318108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59c8104bd0 msgr2=0x7f59c819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c77fe640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59c8104bd0 0x7f59c819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c77fe640 1 -- 172.21.15.102:0/2683318108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c81a1840 con 0x7f59c81022a0 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c6ffd640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59c8104bd0 0x7f59c819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.817+0000 7f59c77fe640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c81022a0 0x7f59c8199c30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b4a0 tx=0x7f59b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.818+0000 7f59c4ff9640 1 -- 172.21.15.102:0/2683318108 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8014070 con 0x7f59c81022a0 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.818+0000 7f59c4ff9640 1 -- 172.21.15.102:0/2683318108 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b8010ac0 con 0x7f59c81022a0 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.818+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c81a1b30 con 0x7f59c81022a0 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.818+0000 7f59c4ff9640 1 -- 172.21.15.102:0/2683318108 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8017c30 con 0x7f59c81022a0 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.818+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c81a2080 con 0x7f59c81022a0 2024-04-24T23:10:42.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.819+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c8108ff0 con 0x7f59c81022a0 2024-04-24T23:10:42.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.820+0000 7f59c4ff9640 1 -- 172.21.15.102:0/2683318108 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f59b8017070 con 0x7f59c81022a0 2024-04-24T23:10:42.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.820+0000 7f59c4ff9640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f59b00a2b90 0x7f59b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:42.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.820+0000 7f59c6ffd640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f59b00a2b90 0x7f59b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:42.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.821+0000 7f59c4ff9640 1 -- 172.21.15.102:0/2683318108 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f59b80ce300 con 0x7f59c81022a0 2024-04-24T23:10:42.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.821+0000 7f59c6ffd640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f59b00a2b90 0x7f59b00a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0058c0 tx=0x7f59bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:42.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:42.825+0000 7f59c4ff9640 1 -- 172.21.15.102:0/2683318108 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59b809b700 con 0x7f59c81022a0 2024-04-24T23:10:43.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.040+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59c802d170 con 0x7f59c81022a0 2024-04-24T23:10:43.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.040+0000 7f59c4ff9640 1 -- 172.21.15.102:0/2683318108 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7f59b809b000 con 0x7f59c81022a0 2024-04-24T23:10:43.043 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:43.043 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T23:10:43.044 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 8 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f59b00a2b90 msgr2=0x7f59b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f59b00a2b90 0x7f59b00a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0058c0 tx=0x7f59bc005850 comp rx=0 tx=0).stop 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c81022a0 msgr2=0x7f59c8199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c81022a0 0x7f59c8199c30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b4a0 tx=0x7f59b800b960 comp rx=0 tx=0).stop 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 shutdown_connections 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f59b00a2b90 0x7f59b00a5050 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59c8107500 0x7f59c81a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59c8104bd0 0x7f59c819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 --2- 172.21.15.102:0/2683318108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59c81022a0 0x7f59c8199c30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.042+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 >> 172.21.15.102:0/2683318108 conn(0x7f59c80fbbf0 msgr2=0x7f59c81019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:43.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.043+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 shutdown_connections 2024-04-24T23:10:43.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:43.043+0000 7f59ce142640 1 -- 172.21.15.102:0/2683318108 wait complete. 2024-04-24T23:10:43.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:43 smithi102 ceph-mon[29216]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:43.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:43 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2683318108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:43.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:43 smithi161 ceph-mon[32709]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:43.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:43 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2683318108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:43.610 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:10:43 smithi161 systemd[1]: Starting Ceph mds.h for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:43.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:43 smithi183 ceph-mon[33137]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:43.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:43 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2683318108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:43.986 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:10:43 smithi161 podman[62683]: 2024-04-24 23:10:43.580401217 +0000 UTC m=+0.018427411 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:43.986 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:10:43 smithi161 podman[62683]: 2024-04-24 23:10:43.707612359 +0000 UTC m=+0.145638539 container create 0edfb28a372e6632af5011b922a033bf600bc23d1cc6052c42f93667bbb940b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:10:43.986 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:10:43 smithi161 podman[62683]: 2024-04-24 23:10:43.985203536 +0000 UTC m=+0.423229713 container init 0edfb28a372e6632af5011b922a033bf600bc23d1cc6052c42f93667bbb940b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T23:10:44.359 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:10:43 smithi161 podman[62683]: 2024-04-24 23:10:43.98785747 +0000 UTC m=+0.425883651 container start 0edfb28a372e6632af5011b922a033bf600bc23d1cc6052c42f93667bbb940b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T23:10:44.360 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:10:44 smithi161 ceph-mds[62702]: -- 172.21.15.161:0/1948648541 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55679c7d3a40 con 0x55679c7a6c00 2024-04-24T23:10:44.360 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:10:44 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h[62698]: starting mds.h at 2024-04-24T23:10:44.360 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:10:44 smithi161 bash[62683]: 0edfb28a372e6632af5011b922a033bf600bc23d1cc6052c42f93667bbb940b4 2024-04-24T23:10:44.360 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:10:44 smithi161 systemd[1]: Started Ceph mds.h for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:45.056 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:45.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.165+0000 7f1617784640 1 -- 172.21.15.102:0/3882471599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610101880 msgr2=0x7f1610101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.165+0000 7f1617784640 1 --2- 172.21.15.102:0/3882471599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610101880 0x7f1610101d00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1600009900 tx=0x7f160002a410 comp rx=0 tx=0).stop 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.165+0000 7f1617784640 1 -- 172.21.15.102:0/3882471599 shutdown_connections 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.165+0000 7f1617784640 1 --2- 172.21.15.102:0/3882471599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102240 0x7f1610108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.165+0000 7f1617784640 1 --2- 172.21.15.102:0/3882471599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610101880 0x7f1610101d00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.165+0000 7f1617784640 1 --2- 172.21.15.102:0/3882471599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1610100690 0x7f1610100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.165+0000 7f1617784640 1 -- 172.21.15.102:0/3882471599 >> 172.21.15.102:0/3882471599 conn(0x7f16100fbc20 msgr2=0x7f16100fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.165+0000 7f1617784640 1 -- 172.21.15.102:0/3882471599 shutdown_connections 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.165+0000 7f1617784640 1 -- 172.21.15.102:0/3882471599 wait complete. 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1617784640 1 Processor -- start 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1617784640 1 -- start start 2024-04-24T23:10:45.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1617784640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610100690 0x7f1610197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1617784640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1610101880 0x7f1610197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1617784640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102240 0x7f161019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1617784640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f161010c230 con 0x7f1610100690 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1617784640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f161010c0b0 con 0x7f1610101880 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1617784640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f161010c3b0 con 0x7f1610102240 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615520640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610100690 0x7f1610197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615520640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610100690 0x7f1610197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53476/0 (socket says 172.21.15.102:53476) 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615520640 1 -- 172.21.15.102:0/924745686 learned_addr learned my addr 172.21.15.102:0/924745686 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615d21640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102240 0x7f161019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1614d1f640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1610101880 0x7f1610197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615520640 1 -- 172.21.15.102:0/924745686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102240 msgr2=0x7f161019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615520640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102240 0x7f161019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615520640 1 -- 172.21.15.102:0/924745686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1610101880 msgr2=0x7f1610197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615520640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1610101880 0x7f1610197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615520640 1 -- 172.21.15.102:0/924745686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161019f670 con 0x7f1610100690 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.166+0000 7f1615520640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610100690 0x7f1610197a50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f160c00b450 tx=0x7f160c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.167+0000 7f16067fc640 1 -- 172.21.15.102:0/924745686 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c015020 con 0x7f1610100690 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.167+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161019f960 con 0x7f1610100690 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.167+0000 7f16067fc640 1 -- 172.21.15.102:0/924745686 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f160c011d40 con 0x7f1610100690 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.167+0000 7f16067fc640 1 -- 172.21.15.102:0/924745686 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c00f340 con 0x7f1610100690 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.167+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161019ff80 con 0x7f1610100690 2024-04-24T23:10:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.168+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16101a01b0 con 0x7f1610100690 2024-04-24T23:10:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.169+0000 7f16067fc640 1 -- 172.21.15.102:0/924745686 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f160c018020 con 0x7f1610100690 2024-04-24T23:10:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.169+0000 7f16067fc640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f15fc0a2b90 0x7f15fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.169+0000 7f1614d1f640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f15fc0a2b90 0x7f15fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.169+0000 7f16067fc640 1 -- 172.21.15.102:0/924745686 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f160c0ce670 con 0x7f1610100690 2024-04-24T23:10:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.170+0000 7f1614d1f640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f15fc0a2b90 0x7f15fc0a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1610198f60 tx=0x7f16000314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:45.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.174+0000 7f16067fc640 1 -- 172.21.15.102:0/924745686 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f160c01f020 con 0x7f1610100690 2024-04-24T23:10:45.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.385+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16101a0380 con 0x7f1610100690 2024-04-24T23:10:45.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.386+0000 7f16067fc640 1 -- 172.21.15.102:0/924745686 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7f160c00d040 con 0x7f1610100690 2024-04-24T23:10:45.388 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:45.388 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:10:45.388 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 9 2024-04-24T23:10:45.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f15fc0a2b90 msgr2=0x7f15fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f15fc0a2b90 0x7f15fc0a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1610198f60 tx=0x7f16000314a0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610100690 msgr2=0x7f1610197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:45.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610100690 0x7f1610197a50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f160c00b450 tx=0x7f160c00b910 comp rx=0 tx=0).stop 2024-04-24T23:10:45.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 shutdown_connections 2024-04-24T23:10:45.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f15fc0a2b90 0x7f15fc0a5050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102240 0x7f161019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1610101880 0x7f1610197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 --2- 172.21.15.102:0/924745686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1610100690 0x7f1610197a50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:45.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 >> 172.21.15.102:0/924745686 conn(0x7f16100fbc20 msgr2=0x7f1610069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:45.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 shutdown_connections 2024-04-24T23:10:45.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:45.388+0000 7f1617784640 1 -- 172.21.15.102:0/924745686 wait complete. 2024-04-24T23:10:45.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:45.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:45.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:45.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:45.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:45.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:45.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: Deploying daemon mds.i on smithi183 2024-04-24T23:10:45.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:45.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] up:boot 2024-04-24T23:10:45.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: fsmap 8 up:standby 2024-04-24T23:10:45.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:45 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:10:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:45.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:45.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: Deploying daemon mds.i on smithi183 2024-04-24T23:10:45.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:45.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] up:boot 2024-04-24T23:10:45.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: fsmap 8 up:standby 2024-04-24T23:10:45.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:45 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:10:45.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:45.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:45.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:45.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:45.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:45.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:45.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: Deploying daemon mds.i on smithi183 2024-04-24T23:10:45.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:45.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] up:boot 2024-04-24T23:10:45.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: fsmap 8 up:standby 2024-04-24T23:10:45.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:45 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:10:46.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:46 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/924745686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:46.480 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:10:46 smithi183 systemd[1]: Starting Ceph mds.i for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:46.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:46 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/924745686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:46.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:46 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/924745686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:46.749 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:10:46 smithi183 podman[63670]: 2024-04-24 23:10:46.381666724 +0000 UTC m=+0.018351962 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:46.749 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:10:46 smithi183 podman[63670]: 2024-04-24 23:10:46.504342226 +0000 UTC m=+0.141027439 container create a1bcedce8d3f3c41a129a04b0da550f37fd40031b07a8c08f7ea89c4645003b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T23:10:46.749 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:10:46 smithi183 podman[63670]: 2024-04-24 23:10:46.748660587 +0000 UTC m=+0.385345806 container init a1bcedce8d3f3c41a129a04b0da550f37fd40031b07a8c08f7ea89c4645003b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:10:47.012 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:10:46 smithi183 podman[63670]: 2024-04-24 23:10:46.751344872 +0000 UTC m=+0.388030089 container start a1bcedce8d3f3c41a129a04b0da550f37fd40031b07a8c08f7ea89c4645003b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T23:10:47.012 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:10:46 smithi183 ceph-mds[63690]: -- 172.21.15.183:0/3781523524 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f03bd3ba40 con 0x55f03bd0f000 2024-04-24T23:10:47.012 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:10:46 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i[63686]: starting mds.i at 2024-04-24T23:10:47.013 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:10:46 smithi183 bash[63670]: a1bcedce8d3f3c41a129a04b0da550f37fd40031b07a8c08f7ea89c4645003b5 2024-04-24T23:10:47.013 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:10:46 smithi183 systemd[1]: Started Ceph mds.i for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:47.403 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:47.425 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:47 smithi102 ceph-mon[29216]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:47.425 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:47.425 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:47.425 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:47.426 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:47.426 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:47.426 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:47 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:47.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.509+0000 7fd790529640 1 -- 172.21.15.102:0/4250107167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788102230 msgr2=0x7fd788108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.509+0000 7fd790529640 1 --2- 172.21.15.102:0/4250107167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788102230 0x7fd788108ac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd780009ca0 tx=0x7fd78002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.509+0000 7fd790529640 1 -- 172.21.15.102:0/4250107167 shutdown_connections 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 --2- 172.21.15.102:0/4250107167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788102230 0x7fd788108ac0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 --2- 172.21.15.102:0/4250107167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd788101870 0x7fd788101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 --2- 172.21.15.102:0/4250107167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd788100680 0x7fd788100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 -- 172.21.15.102:0/4250107167 >> 172.21.15.102:0/4250107167 conn(0x7fd7880fbbf0 msgr2=0x7fd7880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 -- 172.21.15.102:0/4250107167 shutdown_connections 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 -- 172.21.15.102:0/4250107167 wait complete. 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 Processor -- start 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 -- start start 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788100680 0x7fd788069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd788101870 0x7fd78819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd788102230 0x7fd78819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810c1b0 con 0x7fd788100680 2024-04-24T23:10:47.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810c030 con 0x7fd788102230 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd790529640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810c330 con 0x7fd788101870 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd78e2c5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788100680 0x7fd788069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd78e2c5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788100680 0x7fd788069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38366/0 (socket says 172.21.15.102:38366) 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd78e2c5640 1 -- 172.21.15.102:0/418264823 learned_addr learned my addr 172.21.15.102:0/418264823 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.510+0000 7fd78dac4640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd788101870 0x7fd78819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd78eac6640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd788102230 0x7fd78819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd78e2c5640 1 -- 172.21.15.102:0/418264823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd788101870 msgr2=0x7fd78819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd78e2c5640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd788101870 0x7fd78819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd78e2c5640 1 -- 172.21.15.102:0/418264823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd788102230 msgr2=0x7fd78819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd78e2c5640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd788102230 0x7fd78819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd78e2c5640 1 -- 172.21.15.102:0/418264823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78819f590 con 0x7fd788100680 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd78eac6640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd788102230 0x7fd78819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd78e2c5640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788100680 0x7fd788069460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd77800e9c0 tx=0x7fd77800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:47.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd77f7fe640 1 -- 172.21.15.102:0/418264823 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77800cd80 con 0x7fd788100680 2024-04-24T23:10:47.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd77f7fe640 1 -- 172.21.15.102:0/418264823 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7780035f0 con 0x7fd788100680 2024-04-24T23:10:47.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78819f880 con 0x7fd788100680 2024-04-24T23:10:47.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd77f7fe640 1 -- 172.21.15.102:0/418264823 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778010690 con 0x7fd788100680 2024-04-24T23:10:47.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.511+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78819fd80 con 0x7fd788100680 2024-04-24T23:10:47.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.512+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7881079a0 con 0x7fd788100680 2024-04-24T23:10:47.517 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:47 smithi183 ceph-mon[33137]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:47.517 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:47.518 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:47.518 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:47.518 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:47.518 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:47.518 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:47 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:47.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.513+0000 7fd77f7fe640 1 -- 172.21.15.102:0/418264823 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd7780037a0 con 0x7fd788100680 2024-04-24T23:10:47.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.514+0000 7fd77f7fe640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd7740a2b90 0x7fd7740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:47.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.514+0000 7fd78dac4640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd7740a2b90 0x7fd7740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:47.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.514+0000 7fd77f7fe640 1 -- 172.21.15.102:0/418264823 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd7780cf2c0 con 0x7fd788100680 2024-04-24T23:10:47.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.514+0000 7fd78dac4640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd7740a2b90 0x7fd7740a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7880691d0 tx=0x7fd78400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:47.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.517+0000 7fd77f7fe640 1 -- 172.21.15.102:0/418264823 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd77809c6c0 con 0x7fd788100680 2024-04-24T23:10:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:47 smithi161 ceph-mon[32709]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:47.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:47.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:47.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:47 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:47.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.737+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd78802d170 con 0x7fd788100680 2024-04-24T23:10:47.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.738+0000 7fd77f7fe640 1 -- 172.21.15.102:0/418264823 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7fd77809bfc0 con 0x7fd788100680 2024-04-24T23:10:47.740 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:47.740 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:10:47.741 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 10 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd7740a2b90 msgr2=0x7fd7740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd7740a2b90 0x7fd7740a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7880691d0 tx=0x7fd78400a2d0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788100680 msgr2=0x7fd788069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788100680 0x7fd788069460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd77800e9c0 tx=0x7fd77800ee80 comp rx=0 tx=0).stop 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 shutdown_connections 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd7740a2b90 0x7fd7740a5050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd788102230 0x7fd78819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd788101870 0x7fd78819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 --2- 172.21.15.102:0/418264823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd788100680 0x7fd788069460 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.740+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 >> 172.21.15.102:0/418264823 conn(0x7fd7880fbbf0 msgr2=0x7fd788104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.741+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 shutdown_connections 2024-04-24T23:10:47.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:47.741+0000 7fd790529640 1 -- 172.21.15.102:0/418264823 wait complete. 2024-04-24T23:10:48.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:48 smithi183 ceph-mon[33137]: Deploying daemon mds.g on smithi102 2024-04-24T23:10:48.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:48 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] up:boot 2024-04-24T23:10:48.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:48 smithi183 ceph-mon[33137]: fsmap 9 up:standby 2024-04-24T23:10:48.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:10:48.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:48 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/418264823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:48 smithi102 ceph-mon[29216]: Deploying daemon mds.g on smithi102 2024-04-24T23:10:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:48 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] up:boot 2024-04-24T23:10:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:48 smithi102 ceph-mon[29216]: fsmap 9 up:standby 2024-04-24T23:10:48.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:10:48.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:48 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/418264823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:48 smithi161 ceph-mon[32709]: Deploying daemon mds.g on smithi102 2024-04-24T23:10:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:48 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] up:boot 2024-04-24T23:10:48.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:48 smithi161 ceph-mon[32709]: fsmap 9 up:standby 2024-04-24T23:10:48.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:10:48.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:48 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/418264823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:49.164 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:10:49 smithi102 systemd[1]: Starting Ceph mds.g for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:49.459 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:49 smithi102 ceph-mon[29216]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:49.459 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:10:49 smithi102 podman[70938]: 2024-04-24 23:10:49.192952736 +0000 UTC m=+0.017723397 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:49.459 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:10:49 smithi102 podman[70938]: 2024-04-24 23:10:49.325697396 +0000 UTC m=+0.150468043 container create 3f1a557f2ec0e174e60959d71b98f937b08db66dc06e2a80f1b043cfd23356b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T23:10:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:49 smithi183 ceph-mon[33137]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:49 smithi161 ceph-mon[32709]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:49.754 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:49.778 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:10:49 smithi102 podman[70938]: 2024-04-24 23:10:49.595506159 +0000 UTC m=+0.420276807 container init 3f1a557f2ec0e174e60959d71b98f937b08db66dc06e2a80f1b043cfd23356b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-g, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-24T23:10:49.778 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:10:49 smithi102 podman[70938]: 2024-04-24 23:10:49.598631502 +0000 UTC m=+0.423402157 container start 3f1a557f2ec0e174e60959d71b98f937b08db66dc06e2a80f1b043cfd23356b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:10:49.778 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:10:49 smithi102 ceph-mds[70957]: -- 172.21.15.102:0/1961821818 <== mon.0 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55605cba1a40 con 0x55605cb74c00 2024-04-24T23:10:49.778 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:10:49 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-g[70953]: starting mds.g at 2024-04-24T23:10:49.778 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:10:49 smithi102 bash[70938]: 3f1a557f2ec0e174e60959d71b98f937b08db66dc06e2a80f1b043cfd23356b6 2024-04-24T23:10:49.778 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:10:49 smithi102 systemd[1]: Started Ceph mds.g for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.858+0000 7fa806734640 1 -- 172.21.15.102:0/1646489080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800102230 msgr2=0x7fa800108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.858+0000 7fa806734640 1 --2- 172.21.15.102:0/1646489080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800102230 0x7fa800108ac0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009ca0 tx=0x7fa7f802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.858+0000 7fa806734640 1 -- 172.21.15.102:0/1646489080 shutdown_connections 2024-04-24T23:10:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.858+0000 7fa806734640 1 --2- 172.21.15.102:0/1646489080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800102230 0x7fa800108ac0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.858+0000 7fa806734640 1 --2- 172.21.15.102:0/1646489080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa800101870 0x7fa800101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.858+0000 7fa806734640 1 --2- 172.21.15.102:0/1646489080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa800100680 0x7fa800100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.859+0000 7fa806734640 1 -- 172.21.15.102:0/1646489080 >> 172.21.15.102:0/1646489080 conn(0x7fa8000fbbf0 msgr2=0x7fa8000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:49.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.859+0000 7fa806734640 1 -- 172.21.15.102:0/1646489080 shutdown_connections 2024-04-24T23:10:49.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.859+0000 7fa806734640 1 -- 172.21.15.102:0/1646489080 wait complete. 2024-04-24T23:10:49.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.859+0000 7fa806734640 1 Processor -- start 2024-04-24T23:10:49.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.859+0000 7fa806734640 1 -- start start 2024-04-24T23:10:49.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa806734640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800100680 0x7fa800196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:49.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa806734640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa800101870 0x7fa800196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:49.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa806734640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa800102230 0x7fa80019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:49.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa806734640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa80010c200 con 0x7fa800100680 2024-04-24T23:10:49.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa806734640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa80010c080 con 0x7fa800101870 2024-04-24T23:10:49.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa806734640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa80010c380 con 0x7fa800102230 2024-04-24T23:10:49.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800100680 0x7fa800196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:49.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800100680 0x7fa800196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38414/0 (socket says 172.21.15.102:38414) 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fffff640 1 -- 172.21.15.102:0/966204446 learned_addr learned my addr 172.21.15.102:0/966204446 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7ff7fe640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa800101870 0x7fa800196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fffff640 1 -- 172.21.15.102:0/966204446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa800102230 msgr2=0x7fa80019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fffff640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa800102230 0x7fa80019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fffff640 1 -- 172.21.15.102:0/966204446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa800101870 msgr2=0x7fa800196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fffff640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa800101870 0x7fa800196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fffff640 1 -- 172.21.15.102:0/966204446 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80019e490 con 0x7fa800100680 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fffff640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800100680 0x7fa800196870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400d950 tx=0x7fa7f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fd7fa640 1 -- 172.21.15.102:0/966204446 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4017070 con 0x7fa800100680 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80019e780 con 0x7fa800100680 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fd7fa640 1 -- 172.21.15.102:0/966204446 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f4009c30 con 0x7fa800100680 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.860+0000 7fa7fd7fa640 1 -- 172.21.15.102:0/966204446 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f400f620 con 0x7fa800100680 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.861+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80019ec80 con 0x7fa800100680 2024-04-24T23:10:49.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.862+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8001079a0 con 0x7fa800100680 2024-04-24T23:10:49.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.863+0000 7fa7fd7fa640 1 -- 172.21.15.102:0/966204446 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa7f401a070 con 0x7fa800100680 2024-04-24T23:10:49.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.863+0000 7fa7fd7fa640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa7e80a2b90 0x7fa7e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:49.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.864+0000 7fa7ff7fe640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa7e80a2b90 0x7fa7e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:49.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.864+0000 7fa7fd7fa640 1 -- 172.21.15.102:0/966204446 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa7f40ce6a0 con 0x7fa800100680 2024-04-24T23:10:49.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.864+0000 7fa7ff7fe640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa7e80a2b90 0x7fa7e80a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa800197d80 tx=0x7fa7f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:49.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:49.868+0000 7fa7fd7fa640 1 -- 172.21.15.102:0/966204446 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa7f4009de0 con 0x7fa800100680 2024-04-24T23:10:50.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.089+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa80002d170 con 0x7fa800100680 2024-04-24T23:10:50.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.090+0000 7fa7fd7fa640 1 -- 172.21.15.102:0/966204446 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7fa80002d170 con 0x7fa800100680 2024-04-24T23:10:50.092 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:50.092 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:10:50.092 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 10 2024-04-24T23:10:50.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa7e80a2b90 msgr2=0x7fa7e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa7e80a2b90 0x7fa7e80a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa800197d80 tx=0x7fa7f000b390 comp rx=0 tx=0).stop 2024-04-24T23:10:50.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800100680 msgr2=0x7fa800196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800100680 0x7fa800196870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400d950 tx=0x7fa7f400de10 comp rx=0 tx=0).stop 2024-04-24T23:10:50.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 shutdown_connections 2024-04-24T23:10:50.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa7e80a2b90 0x7fa7e80a5050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa800102230 0x7fa80019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa800101870 0x7fa800196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 --2- 172.21.15.102:0/966204446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa800100680 0x7fa800196870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 >> 172.21.15.102:0/966204446 conn(0x7fa8000fbbf0 msgr2=0x7fa800104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:50.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 shutdown_connections 2024-04-24T23:10:50.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:50.092+0000 7fa806734640 1 -- 172.21.15.102:0/966204446 wait complete. 2024-04-24T23:10:51.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:51.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:51.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:51.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:51.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:51.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:51.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: Deploying daemon mds.a on smithi102 2024-04-24T23:10:51.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/966204446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:51.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.102:6838/881198064,v1:172.21.15.102:6839/881198064] up:boot 2024-04-24T23:10:51.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: fsmap 10 up:standby 2024-04-24T23:10:51.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:10:51.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:50 smithi183 ceph-mon[33137]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:51.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:51.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:51.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:51.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:51.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:51.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:51.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: Deploying daemon mds.a on smithi102 2024-04-24T23:10:51.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/966204446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:51.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.102:6838/881198064,v1:172.21.15.102:6839/881198064] up:boot 2024-04-24T23:10:51.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: fsmap 10 up:standby 2024-04-24T23:10:51.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:10:51.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:50 smithi102 ceph-mon[29216]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: Deploying daemon mds.a on smithi102 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/966204446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.102:6838/881198064,v1:172.21.15.102:6839/881198064] up:boot 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: fsmap 10 up:standby 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:10:51.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:50 smithi161 ceph-mon[32709]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:52.046 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:10:51 smithi102 systemd[1]: Starting Ceph mds.a for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:10:52.106 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:52.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.210+0000 7feda155a640 1 -- 172.21.15.102:0/414275537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 msgr2=0x7fed9c109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.210+0000 7feda155a640 1 --2- 172.21.15.102:0/414275537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 0x7fed9c109bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed94009ca0 tx=0x7fed9402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.210+0000 7feda155a640 1 -- 172.21.15.102:0/414275537 shutdown_connections 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.210+0000 7feda155a640 1 --2- 172.21.15.102:0/414275537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 0x7fed9c109bf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.210+0000 7feda155a640 1 --2- 172.21.15.102:0/414275537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed9c1029a0 0x7fed9c102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.210+0000 7feda155a640 1 --2- 172.21.15.102:0/414275537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c1017b0 0x7fed9c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.210+0000 7feda155a640 1 -- 172.21.15.102:0/414275537 >> 172.21.15.102:0/414275537 conn(0x7fed9c0fcd40 msgr2=0x7fed9c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.210+0000 7feda155a640 1 -- 172.21.15.102:0/414275537 shutdown_connections 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.210+0000 7feda155a640 1 -- 172.21.15.102:0/414275537 wait complete. 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7feda155a640 1 Processor -- start 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7feda155a640 1 -- start start 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7feda155a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed9c1017b0 0x7fed9c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7feda155a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c1029a0 0x7fed9c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7feda155a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 0x7fed9c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:52.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7feda155a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c10d350 con 0x7fed9c103360 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7feda155a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c10d1d0 con 0x7fed9c1017b0 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7feda155a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c10d4d0 con 0x7fed9c1029a0 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7fed9b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 0x7fed9c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7fed9b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 0x7fed9c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38434/0 (socket says 172.21.15.102:38434) 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.211+0000 7fed9b7fe640 1 -- 172.21.15.102:0/169164760 learned_addr learned my addr 172.21.15.102:0/169164760 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed9a7fc640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c1029a0 0x7fed9c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed9affd640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed9c1017b0 0x7fed9c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed9b7fe640 1 -- 172.21.15.102:0/169164760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c1029a0 msgr2=0x7fed9c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed9b7fe640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c1029a0 0x7fed9c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed9b7fe640 1 -- 172.21.15.102:0/169164760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed9c1017b0 msgr2=0x7fed9c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed9b7fe640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed9c1017b0 0x7fed9c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed9b7fe640 1 -- 172.21.15.102:0/169164760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c19f640 con 0x7fed9c103360 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed9b7fe640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 0x7fed9c19efd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed94009dc0 tx=0x7fed94003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed83fff640 1 -- 172.21.15.102:0/169164760 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94032ec0 con 0x7fed9c103360 2024-04-24T23:10:52.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed83fff640 1 -- 172.21.15.102:0/169164760 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed940044f0 con 0x7fed9c103360 2024-04-24T23:10:52.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c19f8d0 con 0x7fed9c103360 2024-04-24T23:10:52.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7fed83fff640 1 -- 172.21.15.102:0/169164760 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94030d40 con 0x7fed9c103360 2024-04-24T23:10:52.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.212+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c19fe20 con 0x7fed9c103360 2024-04-24T23:10:52.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.213+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9c108b50 con 0x7fed9c103360 2024-04-24T23:10:52.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.214+0000 7fed83fff640 1 -- 172.21.15.102:0/169164760 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fed94004040 con 0x7fed9c103360 2024-04-24T23:10:52.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.215+0000 7fed83fff640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fed840a2d30 0x7fed840a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:52.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.215+0000 7fed9affd640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fed840a2d30 0x7fed840a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:52.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.215+0000 7fed83fff640 1 -- 172.21.15.102:0/169164760 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fed940f0310 con 0x7fed9c103360 2024-04-24T23:10:52.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.215+0000 7fed9affd640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fed840a2d30 0x7fed840a51f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed9c102800 tx=0x7fed8c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:52.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.219+0000 7fed83fff640 1 -- 172.21.15.102:0/169164760 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fed940bd710 con 0x7fed9c103360 2024-04-24T23:10:52.321 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:10:52 smithi102 podman[71288]: 2024-04-24 23:10:52.112463155 +0000 UTC m=+0.018664349 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:10:52.322 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:10:52 smithi102 podman[71288]: 2024-04-24 23:10:52.231906644 +0000 UTC m=+0.138107835 container create 330491a062d525cc0c430fe410490e5a270ddf457fe6417ffa074c9758833140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:10:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.448+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed9c02d370 con 0x7fed9c103360 2024-04-24T23:10:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.449+0000 7fed83fff640 1 -- 172.21.15.102:0/169164760 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7fed940bd010 con 0x7fed9c103360 2024-04-24T23:10:52.451 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:52.451 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:10:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fed840a2d30 msgr2=0x7fed840a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fed840a2d30 0x7fed840a51f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed9c102800 tx=0x7fed8c009110 comp rx=0 tx=0).stop 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 msgr2=0x7fed9c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 0x7fed9c19efd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed94009dc0 tx=0x7fed94003ba0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 shutdown_connections 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fed840a2d30 0x7fed840a51f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed9c103360 0x7fed9c19efd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c1029a0 0x7fed9c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 --2- 172.21.15.102:0/169164760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed9c1017b0 0x7fed9c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 >> 172.21.15.102:0/169164760 conn(0x7fed9c0fcd40 msgr2=0x7fed9c105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 shutdown_connections 2024-04-24T23:10:52.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:52.451+0000 7feda155a640 1 -- 172.21.15.102:0/169164760 wait complete. 2024-04-24T23:10:52.822 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:10:52 smithi102 podman[71288]: 2024-04-24 23:10:52.460432923 +0000 UTC m=+0.366634119 container init 330491a062d525cc0c430fe410490e5a270ddf457fe6417ffa074c9758833140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T23:10:52.822 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:10:52 smithi102 podman[71288]: 2024-04-24 23:10:52.46346888 +0000 UTC m=+0.369670065 container start 330491a062d525cc0c430fe410490e5a270ddf457fe6417ffa074c9758833140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T23:10:52.822 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:10:52 smithi102 ceph-mds[71358]: -- 172.21.15.102:0/1622522785 <== mon.0 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589026a9a40 con 0x55890267d000 2024-04-24T23:10:52.822 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:10:52 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-a[71350]: starting mds.a at 2024-04-24T23:10:52.822 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:10:52 smithi102 bash[71288]: 330491a062d525cc0c430fe410490e5a270ddf457fe6417ffa074c9758833140 2024-04-24T23:10:52.822 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:10:52 smithi102 systemd[1]: Started Ceph mds.a for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:53 smithi102 ceph-mon[29216]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:53 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/169164760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:53 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:53 smithi161 ceph-mon[32709]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:53.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:53 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/169164760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:53.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:53.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:53.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:53.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:53.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:53.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:53 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:53 smithi183 ceph-mon[33137]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:53 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/169164760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:53.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:53.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:53 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:54.467 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:54.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:54 smithi102 ceph-mon[29216]: Deploying daemon mds.b on smithi161 2024-04-24T23:10:54.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:54 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.102:6840/467071210,v1:172.21.15.102:6841/467071210] up:boot 2024-04-24T23:10:54.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:54 smithi102 ceph-mon[29216]: fsmap 11 up:standby 2024-04-24T23:10:54.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:54 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:10:54.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.574+0000 7f80be076640 1 -- 172.21.15.102:0/2282567357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8105330 msgr2=0x7f80b81057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:54.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.574+0000 7f80be076640 1 --2- 172.21.15.102:0/2282567357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8105330 0x7f80b81057b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009ca0 tx=0x7f80b002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.574+0000 7f80be076640 1 -- 172.21.15.102:0/2282567357 shutdown_connections 2024-04-24T23:10:54.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.574+0000 7f80be076640 1 --2- 172.21.15.102:0/2282567357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8105330 0x7f80b81057b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.574+0000 7f80be076640 1 --2- 172.21.15.102:0/2282567357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80b8102980 0x7f80b8104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.574+0000 7f80be076640 1 --2- 172.21.15.102:0/2282567357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8100010 0x7f80b8102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.574+0000 7f80be076640 1 -- 172.21.15.102:0/2282567357 >> 172.21.15.102:0/2282567357 conn(0x7f80b80fbc30 msgr2=0x7f80b80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:54.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.574+0000 7f80be076640 1 -- 172.21.15.102:0/2282567357 shutdown_connections 2024-04-24T23:10:54.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.574+0000 7f80be076640 1 -- 172.21.15.102:0/2282567357 wait complete. 2024-04-24T23:10:54.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80be076640 1 Processor -- start 2024-04-24T23:10:54.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80be076640 1 -- start start 2024-04-24T23:10:54.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80be076640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8100010 0x7f80b819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:54.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80be076640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80b8102980 0x7f80b819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:54.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80be076640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8105330 0x7f80b81a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:54.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80be076640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8113920 con 0x7f80b8100010 2024-04-24T23:10:54.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80be076640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81137a0 con 0x7f80b8105330 2024-04-24T23:10:54.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80be076640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8113aa0 con 0x7f80b8102980 2024-04-24T23:10:54.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80b77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8100010 0x7f80b819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80b77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8100010 0x7f80b819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38464/0 (socket says 172.21.15.102:38464) 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80b77fe640 1 -- 172.21.15.102:0/1946032266 learned_addr learned my addr 172.21.15.102:0/1946032266 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.575+0000 7f80b6ffd640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80b8102980 0x7f80b819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b7fff640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8105330 0x7f80b81a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b77fe640 1 -- 172.21.15.102:0/1946032266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80b8102980 msgr2=0x7f80b819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b77fe640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80b8102980 0x7f80b819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b77fe640 1 -- 172.21.15.102:0/1946032266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8105330 msgr2=0x7f80b81a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b77fe640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8105330 0x7f80b81a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b77fe640 1 -- 172.21.15.102:0/1946032266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b81a5b80 con 0x7f80b8100010 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b7fff640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8105330 0x7f80b81a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:54.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b77fe640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8100010 0x7f80b819df60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f80a800b4a0 tx=0x7f80a800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:54.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b4ff9640 1 -- 172.21.15.102:0/1946032266 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8015020 con 0x7f80b8100010 2024-04-24T23:10:54.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b4ff9640 1 -- 172.21.15.102:0/1946032266 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80a8011ce0 con 0x7f80b8100010 2024-04-24T23:10:54.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b81a5e70 con 0x7f80b8100010 2024-04-24T23:10:54.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80b4ff9640 1 -- 172.21.15.102:0/1946032266 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8010dc0 con 0x7f80b8100010 2024-04-24T23:10:54.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.576+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b81a62d0 con 0x7f80b8100010 2024-04-24T23:10:54.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.577+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b80fd720 con 0x7f80b8100010 2024-04-24T23:10:54.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.578+0000 7f80b4ff9640 1 -- 172.21.15.102:0/1946032266 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f80a8018020 con 0x7f80b8100010 2024-04-24T23:10:54.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.579+0000 7f80b4ff9640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f80a00a2b90 0x7f80a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:54.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.579+0000 7f80b6ffd640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f80a00a2b90 0x7f80a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:54.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.579+0000 7f80b4ff9640 1 -- 172.21.15.102:0/1946032266 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f80a80ce480 con 0x7f80b8100010 2024-04-24T23:10:54.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.579+0000 7f80b6ffd640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f80a00a2b90 0x7f80a00a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f80b819f470 tx=0x7f80ac00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:54.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.584+0000 7f80b4ff9640 1 -- 172.21.15.102:0/1946032266 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f80a801f020 con 0x7f80b8100010 2024-04-24T23:10:54.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:54 smithi161 ceph-mon[32709]: Deploying daemon mds.b on smithi161 2024-04-24T23:10:54.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:54 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.102:6840/467071210,v1:172.21.15.102:6841/467071210] up:boot 2024-04-24T23:10:54.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:54 smithi161 ceph-mon[32709]: fsmap 11 up:standby 2024-04-24T23:10:54.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:54 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:10:54.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:54 smithi183 ceph-mon[33137]: Deploying daemon mds.b on smithi161 2024-04-24T23:10:54.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:54 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.102:6840/467071210,v1:172.21.15.102:6841/467071210] up:boot 2024-04-24T23:10:54.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:54 smithi183 ceph-mon[33137]: fsmap 11 up:standby 2024-04-24T23:10:54.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:54 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:10:54.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.798+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80b802d170 con 0x7f80b8100010 2024-04-24T23:10:54.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.798+0000 7f80b4ff9640 1 -- 172.21.15.102:0/1946032266 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7f80a809b880 con 0x7f80b8100010 2024-04-24T23:10:54.800 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:54.800 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:10:54.801 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 12 2024-04-24T23:10:54.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f80a00a2b90 msgr2=0x7f80a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:54.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f80a00a2b90 0x7f80a00a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f80b819f470 tx=0x7f80ac00a2d0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8100010 msgr2=0x7f80b819df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:54.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8100010 0x7f80b819df60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f80a800b4a0 tx=0x7f80a800b8a0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 shutdown_connections 2024-04-24T23:10:54.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f80a00a2b90 0x7f80a00a5050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b8105330 0x7f80b81a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80b8102980 0x7f80b819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 --2- 172.21.15.102:0/1946032266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80b8100010 0x7f80b819df60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 >> 172.21.15.102:0/1946032266 conn(0x7f80b80fbc30 msgr2=0x7f80b8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:54.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 shutdown_connections 2024-04-24T23:10:54.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:54.800+0000 7f80be076640 1 -- 172.21.15.102:0/1946032266 wait complete. 2024-04-24T23:10:55.284 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:10:55 smithi161 podman[62993]: 2024-04-24 23:10:55.221566849 +0000 UTC m=+0.137143139 container create 3182ad60e6179bf9662c5f7a1c8cb13d76055465ed961a625cd1ed8630fbb25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:10:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:55 smithi102 ceph-mon[29216]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:55 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1946032266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:55.592 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:55 smithi161 ceph-mon[32709]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:55.592 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:55 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1946032266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:55.592 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:10:55 smithi161 podman[62993]: 2024-04-24 23:10:55.49951972 +0000 UTC m=+0.415096004 container init 3182ad60e6179bf9662c5f7a1c8cb13d76055465ed961a625cd1ed8630fbb25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:10:55.592 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:10:55 smithi161 podman[62993]: 2024-04-24 23:10:55.501987838 +0000 UTC m=+0.417564126 container start 3182ad60e6179bf9662c5f7a1c8cb13d76055465ed961a625cd1ed8630fbb25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T23:10:55.592 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:10:55 smithi161 ceph-mds[63013]: -- 172.21.15.161:0/951925672 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559707ab9a40 con 0x559707a8cc00 2024-04-24T23:10:55.592 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:10:55 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b[63009]: starting mds.b at 2024-04-24T23:10:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:55 smithi183 ceph-mon[33137]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:55 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1946032266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:55.859 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:10:55 smithi161 bash[62993]: 3182ad60e6179bf9662c5f7a1c8cb13d76055465ed961a625cd1ed8630fbb25a 2024-04-24T23:10:55.859 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:10:55 smithi161 systemd[1]: Started Ceph mds.b for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:10:56.814 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:10:56.906 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.906 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.906 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.907 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.907 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.907 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:56.907 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:56.907 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] up:boot 2024-04-24T23:10:56.907 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: fsmap 12 up:standby 2024-04-24T23:10:56.907 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:56 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:10:56.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.914+0000 7faf9d8a5640 1 -- 172.21.15.102:0/99245367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 msgr2=0x7faf98101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.914+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/99245367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 0x7faf98101d70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009900 tx=0x7faf8c02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:56.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.914+0000 7faf9d8a5640 1 -- 172.21.15.102:0/99245367 shutdown_connections 2024-04-24T23:10:56.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.914+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/99245367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf981022b0 0x7faf98108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.914+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/99245367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 0x7faf98101d70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.914+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/99245367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf98100700 0x7faf98100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.914+0000 7faf9d8a5640 1 -- 172.21.15.102:0/99245367 >> 172.21.15.102:0/99245367 conn(0x7faf980fbc90 msgr2=0x7faf980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:56.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.914+0000 7faf9d8a5640 1 -- 172.21.15.102:0/99245367 shutdown_connections 2024-04-24T23:10:56.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.914+0000 7faf9d8a5640 1 -- 172.21.15.102:0/99245367 wait complete. 2024-04-24T23:10:56.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf9d8a5640 1 Processor -- start 2024-04-24T23:10:56.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf9d8a5640 1 -- start start 2024-04-24T23:10:56.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf9d8a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf98100700 0x7faf98196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf9d8a5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 0x7faf98196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf9d8a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf981022b0 0x7faf9819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf9d8a5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faf9810c1a0 con 0x7faf981018f0 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf9d8a5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faf9810c020 con 0x7faf98100700 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf9d8a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf9810c320 con 0x7faf981022b0 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf967fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 0x7faf98196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf967fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 0x7faf98196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40194/0 (socket says 172.21.15.102:40194) 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf967fc640 1 -- 172.21.15.102:0/1616612802 learned_addr learned my addr 172.21.15.102:0/1616612802 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf96ffd640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf98100700 0x7faf98196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf977fe640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf981022b0 0x7faf9819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf967fc640 1 -- 172.21.15.102:0/1616612802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf981022b0 msgr2=0x7faf9819de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf967fc640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf981022b0 0x7faf9819de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf967fc640 1 -- 172.21.15.102:0/1616612802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf98100700 msgr2=0x7faf98196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf967fc640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf98100700 0x7faf98196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.915+0000 7faf967fc640 1 -- 172.21.15.102:0/1616612802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9819e490 con 0x7faf981018f0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.916+0000 7faf967fc640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 0x7faf98196db0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf8c005940 tx=0x7faf8c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.916+0000 7faf9c8a3640 1 -- 172.21.15.102:0/1616612802 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c03f030 con 0x7faf981018f0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.916+0000 7faf9c8a3640 1 -- 172.21.15.102:0/1616612802 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf8c003cb0 con 0x7faf981018f0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.916+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9819e720 con 0x7faf981018f0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.916+0000 7faf9c8a3640 1 -- 172.21.15.102:0/1616612802 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c035070 con 0x7faf981018f0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.916+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9819ec10 con 0x7faf981018f0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.917+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf98107aa0 con 0x7faf981018f0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.918+0000 7faf9c8a3640 1 -- 172.21.15.102:0/1616612802 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7faf8c042030 con 0x7faf981018f0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.918+0000 7faf9c8a3640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7faf800a2b90 0x7faf800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.918+0000 7faf96ffd640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7faf800a2b90 0x7faf800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.918+0000 7faf9c8a3640 1 -- 172.21.15.102:0/1616612802 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7faf8c0ef590 con 0x7faf981018f0 2024-04-24T23:10:56.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.919+0000 7faf96ffd640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7faf800a2b90 0x7faf800a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf98101750 tx=0x7faf88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:56.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:56.923+0000 7faf9c8a3640 1 -- 172.21.15.102:0/1616612802 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faf8c0bc340 con 0x7faf981018f0 2024-04-24T23:10:56.931 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.931 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.932 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.932 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.932 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.932 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:56.932 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:56.932 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] up:boot 2024-04-24T23:10:56.932 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: fsmap 12 up:standby 2024-04-24T23:10:56.932 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:56 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:10:56.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.954 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:56.955 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:56.955 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:56.955 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] up:boot 2024-04-24T23:10:56.955 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: fsmap 12 up:standby 2024-04-24T23:10:56.955 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:56 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:10:57.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.138+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf9802d170 con 0x7faf981018f0 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.139+0000 7faf9c8a3640 1 -- 172.21.15.102:0/1616612802 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10497 (secure 0 0 0) 0x7faf8c0c0640 con 0x7faf981018f0 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1258581930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1258581930},{"type":"v1","addr":"172.21.15.161:6839","nonce":1258581930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-24T23:10:57.140 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 13 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7faf800a2b90 msgr2=0x7faf800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7faf800a2b90 0x7faf800a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf98101750 tx=0x7faf88005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 msgr2=0x7faf98196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 0x7faf98196db0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf8c005940 tx=0x7faf8c004430 comp rx=0 tx=0).stop 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 shutdown_connections 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7faf800a2b90 0x7faf800a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf981022b0 0x7faf9819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf981018f0 0x7faf98196db0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 --2- 172.21.15.102:0/1616612802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf98100700 0x7faf98196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 >> 172.21.15.102:0/1616612802 conn(0x7faf980fbc90 msgr2=0x7faf98104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 shutdown_connections 2024-04-24T23:10:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.141+0000 7faf9d8a5640 1 -- 172.21.15.102:0/1616612802 wait complete. 2024-04-24T23:10:57.154 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:10:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 -- 172.21.15.102:0/791399082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a44101300 msgr2=0x7f3a44108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/791399082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a44101300 0x7f3a44108ba0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009be0 tx=0x7f3a3c02a6c0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 -- 172.21.15.102:0/791399082 shutdown_connections 2024-04-24T23:10:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/791399082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a44101300 0x7f3a44108ba0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/791399082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a44100940 0x7f3a44100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/791399082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a44101c10 0x7f3a44102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 -- 172.21.15.102:0/791399082 >> 172.21.15.102:0/791399082 conn(0x7f3a440fb370 msgr2=0x7f3a440fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 -- 172.21.15.102:0/791399082 shutdown_connections 2024-04-24T23:10:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 -- 172.21.15.102:0/791399082 wait complete. 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.263+0000 7f3a48b3b640 1 Processor -- start 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a48b3b640 1 -- start start 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a48b3b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a44100940 0x7f3a44197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a48b3b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a44101300 0x7f3a44197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a48b3b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a44101c10 0x7f3a4419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a48b3b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4410c1c0 con 0x7f3a44100940 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a48b3b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4410c040 con 0x7f3a44101300 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a48b3b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4410c340 con 0x7f3a44101c10 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a4259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a44100940 0x7f3a44197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a4259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a44100940 0x7f3a44197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40210/0 (socket says 172.21.15.102:40210) 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a4259c640 1 -- 172.21.15.102:0/1253825937 learned_addr learned my addr 172.21.15.102:0/1253825937 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a41d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a44101300 0x7f3a44197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a42d9d640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a44101c10 0x7f3a4419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a4259c640 1 -- 172.21.15.102:0/1253825937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a44101c10 msgr2=0x7f3a4419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a4259c640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a44101c10 0x7f3a4419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a4259c640 1 -- 172.21.15.102:0/1253825937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a44101300 msgr2=0x7f3a44197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a4259c640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a44101300 0x7f3a44197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a4259c640 1 -- 172.21.15.102:0/1253825937 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4419f5a0 con 0x7f3a44100940 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a4259c640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a44100940 0x7f3a44197980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400e9c0 tx=0x7f3a3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.264+0000 7f3a337fe640 1 -- 172.21.15.102:0/1253825937 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3400cd80 con 0x7f3a44100940 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.265+0000 7f3a337fe640 1 -- 172.21.15.102:0/1253825937 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a340035f0 con 0x7f3a44100940 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.265+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4419f890 con 0x7f3a44100940 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.265+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4419fde0 con 0x7f3a44100940 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.265+0000 7f3a337fe640 1 -- 172.21.15.102:0/1253825937 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34010690 con 0x7f3a44100940 2024-04-24T23:10:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.266+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a44067cd0 con 0x7f3a44100940 2024-04-24T23:10:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.267+0000 7f3a337fe640 1 -- 172.21.15.102:0/1253825937 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3a340037a0 con 0x7f3a44100940 2024-04-24T23:10:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.267+0000 7f3a337fe640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3a2c0a2c10 0x7f3a2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.268+0000 7f3a337fe640 1 -- 172.21.15.102:0/1253825937 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3a340cf240 con 0x7f3a44100940 2024-04-24T23:10:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.268+0000 7f3a41d9b640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3a2c0a2c10 0x7f3a2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.268+0000 7f3a41d9b640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3a2c0a2c10 0x7f3a2c0a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a44198e90 tx=0x7f3a38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.272+0000 7f3a337fe640 1 -- 172.21.15.102:0/1253825937 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3a3409c640 con 0x7f3a44100940 2024-04-24T23:10:57.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.500+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a441a02f0 con 0x7f3a44100940 2024-04-24T23:10:57.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.501+0000 7f3a337fe640 1 -- 172.21.15.102:0/1253825937 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18516 (secure 0 0 0) 0x7f3a3409bf40 con 0x7f3a44100940 2024-04-24T23:10:57.502 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:57.502 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":82,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:10:08.965212+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3a2c0a2c10 msgr2=0x7f3a2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3a2c0a2c10 0x7f3a2c0a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a44198e90 tx=0x7f3a38009260 comp rx=0 tx=0).stop 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a44100940 msgr2=0x7f3a44197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a44100940 0x7f3a44197980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400e9c0 tx=0x7f3a3400ee80 comp rx=0 tx=0).stop 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 shutdown_connections 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3a2c0a2c10 0x7f3a2c0a50d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a44101c10 0x7f3a4419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a44101300 0x7f3a44197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 --2- 172.21.15.102:0/1253825937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a44100940 0x7f3a44197980 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 >> 172.21.15.102:0/1253825937 conn(0x7f3a440fb370 msgr2=0x7f3a440fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 shutdown_connections 2024-04-24T23:10:57.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.503+0000 7f3a48b3b640 1 -- 172.21.15.102:0/1253825937 wait complete. 2024-04-24T23:10:57.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:10:57.516 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:10:57.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.639+0000 7feca37dc640 1 -- 172.21.15.102:0/1393506271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c101c20 msgr2=0x7fec9c102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.639+0000 7feca37dc640 1 --2- 172.21.15.102:0/1393506271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c101c20 0x7fec9c102040 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec9802a410 comp rx=0 tx=0).stop 2024-04-24T23:10:57.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.639+0000 7feca37dc640 1 -- 172.21.15.102:0/1393506271 shutdown_connections 2024-04-24T23:10:57.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.639+0000 7feca37dc640 1 --2- 172.21.15.102:0/1393506271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec9c101310 0x7fec9c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.639+0000 7feca37dc640 1 --2- 172.21.15.102:0/1393506271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c100950 0x7fec9c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.639+0000 7feca37dc640 1 --2- 172.21.15.102:0/1393506271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c101c20 0x7fec9c102040 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.639+0000 7feca37dc640 1 -- 172.21.15.102:0/1393506271 >> 172.21.15.102:0/1393506271 conn(0x7fec9c0fb380 msgr2=0x7fec9c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.639+0000 7feca37dc640 1 -- 172.21.15.102:0/1393506271 shutdown_connections 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.639+0000 7feca37dc640 1 -- 172.21.15.102:0/1393506271 wait complete. 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca37dc640 1 Processor -- start 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca37dc640 1 -- start start 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca37dc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c100950 0x7fec9c100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca37dc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec9c101310 0x7fec9c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca37dc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c101c20 0x7fec9c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca37dc640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c10c1d0 con 0x7fec9c100950 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca37dc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c10c050 con 0x7fec9c101c20 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca37dc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c10c350 con 0x7fec9c101310 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1578640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c100950 0x7fec9c100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1578640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c100950 0x7fec9c100410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40232/0 (socket says 172.21.15.102:40232) 2024-04-24T23:10:57.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1578640 1 -- 172.21.15.102:0/1219976325 learned_addr learned my addr 172.21.15.102:0/1219976325 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca0d77640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec9c101310 0x7fec9c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1d79640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c101c20 0x7fec9c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1578640 1 -- 172.21.15.102:0/1219976325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec9c101310 msgr2=0x7fec9c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1578640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec9c101310 0x7fec9c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1578640 1 -- 172.21.15.102:0/1219976325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c101c20 msgr2=0x7fec9c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1578640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c101c20 0x7fec9c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1578640 1 -- 172.21.15.102:0/1219976325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c19b400 con 0x7fec9c100950 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca0d77640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec9c101310 0x7fec9c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.640+0000 7feca1578640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c100950 0x7fec9c100410 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fec980314e0 tx=0x7fec98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.641+0000 7fec927fc640 1 -- 172.21.15.102:0/1219976325 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9803f030 con 0x7fec9c100950 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.641+0000 7fec927fc640 1 -- 172.21.15.102:0/1219976325 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec98003cb0 con 0x7fec9c100950 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.641+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9c19b690 con 0x7fec9c100950 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.641+0000 7fec927fc640 1 -- 172.21.15.102:0/1219976325 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98035070 con 0x7fec9c100950 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.641+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9c19bbd0 con 0x7fec9c100950 2024-04-24T23:10:57.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.641+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec9c19c5f0 con 0x7fec9c100950 2024-04-24T23:10:57.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.643+0000 7fec927fc640 1 -- 172.21.15.102:0/1219976325 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fec98042030 con 0x7fec9c100950 2024-04-24T23:10:57.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.643+0000 7fec927fc640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec880a2c10 0x7fec880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.643+0000 7feca0d77640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec880a2c10 0x7fec880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.643+0000 7fec927fc640 1 -- 172.21.15.102:0/1219976325 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fec980ef640 con 0x7fec9c100950 2024-04-24T23:10:57.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.644+0000 7feca0d77640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec880a2c10 0x7fec880a50d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0058c0 tx=0x7fec8c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.647+0000 7fec927fc640 1 -- 172.21.15.102:0/1219976325 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fec980bca40 con 0x7fec9c100950 2024-04-24T23:10:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:57 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1616612802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:57 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1253825937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:57 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:57.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.831+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec9c107820 con 0x7fec9c100950 2024-04-24T23:10:57.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.831+0000 7fec927fc640 1 -- 172.21.15.102:0/1219976325 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fec980bc340 con 0x7fec9c100950 2024-04-24T23:10:57.832 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:10:57.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec880a2c10 msgr2=0x7fec880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec880a2c10 0x7fec880a50d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0058c0 tx=0x7fec8c005850 comp rx=0 tx=0).stop 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c100950 msgr2=0x7fec9c100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c100950 0x7fec9c100410 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fec980314e0 tx=0x7fec98004270 comp rx=0 tx=0).stop 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 shutdown_connections 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fec880a2c10 0x7fec880a50d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9c101c20 0x7fec9c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec9c101310 0x7fec9c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 --2- 172.21.15.102:0/1219976325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec9c100950 0x7fec9c100410 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.833+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 >> 172.21.15.102:0/1219976325 conn(0x7fec9c0fb380 msgr2=0x7fec9c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.834+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 shutdown_connections 2024-04-24T23:10:57.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.834+0000 7feca37dc640 1 -- 172.21.15.102:0/1219976325 wait complete. 2024-04-24T23:10:57.846 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:10:57.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.945+0000 7f40755c8640 1 -- 172.21.15.102:0/2572027896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40701006a0 msgr2=0x7f4070100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.945+0000 7f40755c8640 1 --2- 172.21.15.102:0/2572027896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40701006a0 0x7f4070100ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4064009900 tx=0x7f406402a410 comp rx=0 tx=0).stop 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.945+0000 7f40755c8640 1 -- 172.21.15.102:0/2572027896 shutdown_connections 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.945+0000 7f40755c8640 1 --2- 172.21.15.102:0/2572027896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4070102250 0x7f4070108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.945+0000 7f40755c8640 1 --2- 172.21.15.102:0/2572027896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4070101890 0x7f4070101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.945+0000 7f40755c8640 1 --2- 172.21.15.102:0/2572027896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40701006a0 0x7f4070100ac0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.945+0000 7f40755c8640 1 -- 172.21.15.102:0/2572027896 >> 172.21.15.102:0/2572027896 conn(0x7f40700fbc30 msgr2=0x7f40700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 -- 172.21.15.102:0/2572027896 shutdown_connections 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 -- 172.21.15.102:0/2572027896 wait complete. 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 Processor -- start 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 -- start start 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40701006a0 0x7f4070199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4070101890 0x7f407019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4070102250 0x7f40701a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f407010c1a0 con 0x7f4070102250 2024-04-24T23:10:57.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f407010c020 con 0x7f40701006a0 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.946+0000 7f40755c8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f407010c320 con 0x7f4070101890 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4070102250 0x7f40701a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4070102250 0x7f40701a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40258/0 (socket says 172.21.15.102:40258) 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f406f7fe640 1 -- 172.21.15.102:0/1637362145 learned_addr learned my addr 172.21.15.102:0/1637362145 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f406f7fe640 1 -- 172.21.15.102:0/1637362145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4070101890 msgr2=0x7f407019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f406f7fe640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4070101890 0x7f407019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f406f7fe640 1 -- 172.21.15.102:0/1637362145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40701006a0 msgr2=0x7f4070199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f406f7fe640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40701006a0 0x7f4070199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f406f7fe640 1 -- 172.21.15.102:0/1637362145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40701a1840 con 0x7f4070102250 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f406f7fe640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4070102250 0x7f40701a11a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f406800eaa0 tx=0x7f406800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f4057fff640 1 -- 172.21.15.102:0/1637362145 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406800cdb0 con 0x7f4070102250 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f4057fff640 1 -- 172.21.15.102:0/1637362145 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4068005020 con 0x7f4070102250 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f4057fff640 1 -- 172.21.15.102:0/1637362145 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680116d0 con 0x7f4070102250 2024-04-24T23:10:57.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40701a1b30 con 0x7f4070102250 2024-04-24T23:10:57.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.947+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40701a2080 con 0x7f4070102250 2024-04-24T23:10:57.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.948+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40701a22b0 con 0x7f4070102250 2024-04-24T23:10:57.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.950+0000 7f4057fff640 1 -- 172.21.15.102:0/1637362145 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f40680051d0 con 0x7f4070102250 2024-04-24T23:10:57.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.951+0000 7f4057fff640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f40580a2c10 0x7f40580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:57.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.951+0000 7f406effd640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f40580a2c10 0x7f40580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:57.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.951+0000 7f4057fff640 1 -- 172.21.15.102:0/1637362145 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f40680ceed0 con 0x7f4070102250 2024-04-24T23:10:57.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.951+0000 7f406effd640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f40580a2c10 0x7f40580a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f406402a930 tx=0x7f40640314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:57.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:57.955+0000 7f4057fff640 1 -- 172.21.15.102:0/1637362145 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4068016030 con 0x7f4070102250 2024-04-24T23:10:58.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:57 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1616612802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:58.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:57 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1253825937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:58.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:58.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:58.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:58.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:57 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:58.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:57 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1616612802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:10:58.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:57 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1253825937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:10:58.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:58.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:58.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:58.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:57 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.143+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f40701a2b50 con 0x7f4070102250 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.143+0000 7f4057fff640 1 -- 172.21.15.102:0/1637362145 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f406809c230 con 0x7f4070102250 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi102.stdout:[ 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:10:58.144 INFO:teuthology.orchestra.run.smithi102.stdout: "poolnum": 1, 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi102.stdout: "poolname": ".mgr" 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:10:58.145 INFO:teuthology.orchestra.run.smithi102.stdout:] 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f40580a2c10 msgr2=0x7f40580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f40580a2c10 0x7f40580a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f406402a930 tx=0x7f40640314a0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4070102250 msgr2=0x7f40701a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4070102250 0x7f40701a11a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f406800eaa0 tx=0x7f406800ef60 comp rx=0 tx=0).stop 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 shutdown_connections 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f40580a2c10 0x7f40580a50d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4070102250 0x7f40701a11a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4070101890 0x7f407019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 --2- 172.21.15.102:0/1637362145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40701006a0 0x7f4070199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 >> 172.21.15.102:0/1637362145 conn(0x7f40700fbc30 msgr2=0x7f4070069db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 shutdown_connections 2024-04-24T23:10:58.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.145+0000 7f40755c8640 1 -- 172.21.15.102:0/1637362145 wait complete. 2024-04-24T23:10:58.158 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:10:58.158 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:10:58.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 -- 172.21.15.102:0/2865482842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78103340 msgr2=0x7ffa78109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:58.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/2865482842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78103340 0x7ffa78109bd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009ca0 tx=0x7ffa7002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 -- 172.21.15.102:0/2865482842 shutdown_connections 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/2865482842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78103340 0x7ffa78109bd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/2865482842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa78102980 0x7ffa78102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/2865482842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa78101790 0x7ffa78101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 -- 172.21.15.102:0/2865482842 >> 172.21.15.102:0/2865482842 conn(0x7ffa780fcd20 msgr2=0x7ffa780ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 -- 172.21.15.102:0/2865482842 shutdown_connections 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 -- 172.21.15.102:0/2865482842 wait complete. 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 Processor -- start 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.265+0000 7ffa7efd5640 1 -- start start 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7efd5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa78101790 0x7ffa78197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7efd5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78102980 0x7ffa78197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7efd5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa78103340 0x7ffa7819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:58.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7efd5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7810d2c0 con 0x7ffa78102980 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7efd5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7810d140 con 0x7ffa78101790 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7efd5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7810d440 con 0x7ffa78103340 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa77fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78102980 0x7ffa78197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa77fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78102980 0x7ffa78197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40270/0 (socket says 172.21.15.102:40270) 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa77fff640 1 -- 172.21.15.102:0/1684727205 learned_addr learned my addr 172.21.15.102:0/1684727205 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7d572640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa78103340 0x7ffa7819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7cd71640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa78101790 0x7ffa78197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa77fff640 1 -- 172.21.15.102:0/1684727205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa78103340 msgr2=0x7ffa7819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa77fff640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa78103340 0x7ffa7819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa77fff640 1 -- 172.21.15.102:0/1684727205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa78101790 msgr2=0x7ffa78197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa77fff640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa78101790 0x7ffa78197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa77fff640 1 -- 172.21.15.102:0/1684727205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7819f520 con 0x7ffa78102980 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7d572640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa78103340 0x7ffa7819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:58.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa77fff640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78102980 0x7ffa78197e40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c920 tx=0x7ffa6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:58.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa75ffb640 1 -- 172.21.15.102:0/1684727205 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64007c70 con 0x7ffa78102980 2024-04-24T23:10:58.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa75ffb640 1 -- 172.21.15.102:0/1684727205 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa64007e10 con 0x7ffa78102980 2024-04-24T23:10:58.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7819f810 con 0x7ffa78102980 2024-04-24T23:10:58.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa75ffb640 1 -- 172.21.15.102:0/1684727205 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64018820 con 0x7ffa78102980 2024-04-24T23:10:58.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.266+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7819fd60 con 0x7ffa78102980 2024-04-24T23:10:58.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.267+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78108ab0 con 0x7ffa78102980 2024-04-24T23:10:58.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.268+0000 7ffa75ffb640 1 -- 172.21.15.102:0/1684727205 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ffa640189c0 con 0x7ffa78102980 2024-04-24T23:10:58.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.269+0000 7ffa75ffb640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffa680a2b90 0x7ffa680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:58.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.269+0000 7ffa7cd71640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffa680a2b90 0x7ffa680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:58.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.269+0000 7ffa75ffb640 1 -- 172.21.15.102:0/1684727205 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffa640cedc0 con 0x7ffa78102980 2024-04-24T23:10:58.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.269+0000 7ffa7cd71640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffa680a2b90 0x7ffa680a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffa781027e0 tx=0x7ffa6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:58.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.273+0000 7ffa75ffb640 1 -- 172.21.15.102:0/1684727205 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffa64014030 con 0x7ffa78102980 2024-04-24T23:10:58.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:58.483+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ffa7802d390 con 0x7ffa78102980 2024-04-24T23:10:59.011 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.011 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.011 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1219976325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:59.011 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1637362145' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:10:59.011 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:59.011 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.011 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:59.011 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:10:58 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1684727205' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1219976325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1637362145' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:59.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:10:58 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1684727205' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:10:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1219976325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:10:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1637362145' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:10:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:10:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:59.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:59.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:10:59.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:59.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:10:58 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1684727205' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:10:59.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.282+0000 7ffa75ffb640 1 -- 172.21.15.102:0/1684727205 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7ffa6409c1c0 con 0x7ffa78102980 2024-04-24T23:10:59.283 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' created 2024-04-24T23:10:59.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.284+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffa680a2b90 msgr2=0x7ffa680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.284+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffa680a2b90 0x7ffa680a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffa781027e0 tx=0x7ffa6c005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.284+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78102980 msgr2=0x7ffa78197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.284+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78102980 0x7ffa78197e40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c920 tx=0x7ffa6400cde0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.285+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 shutdown_connections 2024-04-24T23:10:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.285+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffa680a2b90 0x7ffa680a5050 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffa781027e0 tx=0x7ffa6c005f90 comp rx=0 tx=0).stop 2024-04-24T23:10:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.285+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa78103340 0x7ffa7819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.285+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa78102980 0x7ffa78197e40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.285+0000 7ffa7efd5640 1 --2- 172.21.15.102:0/1684727205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa78101790 0x7ffa78197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.285+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 >> 172.21.15.102:0/1684727205 conn(0x7ffa780fcd20 msgr2=0x7ffa78105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:59.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.285+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 shutdown_connections 2024-04-24T23:10:59.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.285+0000 7ffa7efd5640 1 -- 172.21.15.102:0/1684727205 wait complete. 2024-04-24T23:10:59.301 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:10:59.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.409+0000 7f8da45b7640 1 -- 172.21.15.102:0/3702077120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c104b90 msgr2=0x7f8d9c106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:59.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.409+0000 7f8da45b7640 1 --2- 172.21.15.102:0/3702077120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c104b90 0x7f8d9c106f80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009900 tx=0x7f8d8c02a410 comp rx=0 tx=0).stop 2024-04-24T23:10:59.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.409+0000 7f8da45b7640 1 -- 172.21.15.102:0/3702077120 shutdown_connections 2024-04-24T23:10:59.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.409+0000 7f8da45b7640 1 --2- 172.21.15.102:0/3702077120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d9c1074c0 0x7f8d9c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.409+0000 7f8da45b7640 1 --2- 172.21.15.102:0/3702077120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c104b90 0x7f8d9c106f80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.409+0000 7f8da45b7640 1 --2- 172.21.15.102:0/3702077120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c102260 0x7f8d9c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.409+0000 7f8da45b7640 1 -- 172.21.15.102:0/3702077120 >> 172.21.15.102:0/3702077120 conn(0x7f8d9c0fbbf0 msgr2=0x7f8d9c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.409+0000 7f8da45b7640 1 -- 172.21.15.102:0/3702077120 shutdown_connections 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.409+0000 7f8da45b7640 1 -- 172.21.15.102:0/3702077120 wait complete. 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 Processor -- start 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 -- start start 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c102260 0x7f8d9c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d9c104b90 0x7f8d9c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c1074c0 0x7f8d9c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c10d2b0 con 0x7f8d9c1074c0 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c10d130 con 0x7f8d9c102260 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c10d430 con 0x7f8d9c104b90 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2b54640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c1074c0 0x7f8d9c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2b54640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c1074c0 0x7f8d9c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40282/0 (socket says 172.21.15.102:40282) 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2b54640 1 -- 172.21.15.102:0/1153735689 learned_addr learned my addr 172.21.15.102:0/1153735689 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:10:59.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2353640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c102260 0x7f8d9c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da1b52640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d9c104b90 0x7f8d9c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2b54640 1 -- 172.21.15.102:0/1153735689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d9c104b90 msgr2=0x7f8d9c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2b54640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d9c104b90 0x7f8d9c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2b54640 1 -- 172.21.15.102:0/1153735689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c102260 msgr2=0x7f8d9c197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2b54640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c102260 0x7f8d9c197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2b54640 1 -- 172.21.15.102:0/1153735689 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c19f5a0 con 0x7f8d9c1074c0 2024-04-24T23:10:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da2b54640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c1074c0 0x7f8d9c19ef30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400eb10 tx=0x7f8d9400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8d937fe640 1 -- 172.21.15.102:0/1153735689 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400ce20 con 0x7f8d9c1074c0 2024-04-24T23:10:59.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8d937fe640 1 -- 172.21.15.102:0/1153735689 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d94005020 con 0x7f8d9c1074c0 2024-04-24T23:10:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9c19f890 con 0x7f8d9c1074c0 2024-04-24T23:10:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8d937fe640 1 -- 172.21.15.102:0/1153735689 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94010690 con 0x7f8d9c1074c0 2024-04-24T23:10:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.410+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9c19fdb0 con 0x7f8d9c1074c0 2024-04-24T23:10:59.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.411+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d9c0fd6e0 con 0x7f8d9c1074c0 2024-04-24T23:10:59.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.413+0000 7f8d937fe640 1 -- 172.21.15.102:0/1153735689 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8d940051d0 con 0x7f8d9c1074c0 2024-04-24T23:10:59.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.413+0000 7f8d937fe640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8d880a2b90 0x7f8d880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:59.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.413+0000 7f8da2353640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8d880a2b90 0x7f8d880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:59.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.413+0000 7f8d937fe640 1 -- 172.21.15.102:0/1153735689 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f8d940cf7f0 con 0x7f8d9c1074c0 2024-04-24T23:10:59.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.414+0000 7f8da2353640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8d880a2b90 0x7f8d880a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800abe0 tx=0x7f8d9800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:59.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.417+0000 7f8d937fe640 1 -- 172.21.15.102:0/1153735689 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8d9409ca80 con 0x7f8d9c1074c0 2024-04-24T23:10:59.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:10:59.595+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8d9c02d170 con 0x7f8d9c1074c0 2024-04-24T23:11:00.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.289+0000 7f8d937fe640 1 -- 172.21.15.102:0/1153735689 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f8d9409c380 con 0x7f8d9c1074c0 2024-04-24T23:11:00.290 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' created 2024-04-24T23:11:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8d880a2b90 msgr2=0x7f8d880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8d880a2b90 0x7f8d880a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800abe0 tx=0x7f8d9800a2d0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c1074c0 msgr2=0x7f8d9c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c1074c0 0x7f8d9c19ef30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400eb10 tx=0x7f8d9400ef10 comp rx=0 tx=0).stop 2024-04-24T23:11:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 shutdown_connections 2024-04-24T23:11:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8d880a2b90 0x7f8d880a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c1074c0 0x7f8d9c19ef30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d9c104b90 0x7f8d9c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 --2- 172.21.15.102:0/1153735689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c102260 0x7f8d9c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.292+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 >> 172.21.15.102:0/1153735689 conn(0x7f8d9c0fbbf0 msgr2=0x7f8d9c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:00.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.293+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 shutdown_connections 2024-04-24T23:11:00.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.293+0000 7f8da45b7640 1 -- 172.21.15.102:0/1153735689 wait complete. 2024-04-24T23:11:00.317 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:11:00.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.418+0000 7f64421d5640 1 -- 172.21.15.102:0/1920964978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c102250 msgr2=0x7f643c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.418+0000 7f64421d5640 1 --2- 172.21.15.102:0/1920964978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c102250 0x7f643c108ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6434009ca0 tx=0x7f643402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.418+0000 7f64421d5640 1 -- 172.21.15.102:0/1920964978 shutdown_connections 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.418+0000 7f64421d5640 1 --2- 172.21.15.102:0/1920964978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c102250 0x7f643c108ae0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.418+0000 7f64421d5640 1 --2- 172.21.15.102:0/1920964978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c101890 0x7f643c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.418+0000 7f64421d5640 1 --2- 172.21.15.102:0/1920964978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c1006a0 0x7f643c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.418+0000 7f64421d5640 1 -- 172.21.15.102:0/1920964978 >> 172.21.15.102:0/1920964978 conn(0x7f643c0fbc30 msgr2=0x7f643c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.418+0000 7f64421d5640 1 -- 172.21.15.102:0/1920964978 shutdown_connections 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.418+0000 7f64421d5640 1 -- 172.21.15.102:0/1920964978 wait complete. 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f64421d5640 1 Processor -- start 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f64421d5640 1 -- start start 2024-04-24T23:11:00.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f64421d5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c1006a0 0x7f643c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f64421d5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101890 0x7f643c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f64421d5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c102250 0x7f643c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f64421d5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d350 con 0x7f643c101890 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f64421d5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d1d0 con 0x7f643c1006a0 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f64421d5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d4d0 con 0x7f643c102250 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f643affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101890 0x7f643c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f643affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101890 0x7f643c197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40312/0 (socket says 172.21.15.102:40312) 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f643affd640 1 -- 172.21.15.102:0/3421871274 learned_addr learned my addr 172.21.15.102:0/3421871274 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f643b7fe640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c1006a0 0x7f643c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.419+0000 7f643bfff640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c102250 0x7f643c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f643affd640 1 -- 172.21.15.102:0/3421871274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c102250 msgr2=0x7f643c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f643affd640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c102250 0x7f643c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f643affd640 1 -- 172.21.15.102:0/3421871274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c1006a0 msgr2=0x7f643c197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f643affd640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c1006a0 0x7f643c197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:00.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f643affd640 1 -- 172.21.15.102:0/3421871274 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c19f580 con 0x7f643c101890 2024-04-24T23:11:00.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f643affd640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101890 0x7f643c197ea0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f643000e9f0 tx=0x7f643000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:00.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f6438ff9640 1 -- 172.21.15.102:0/3421871274 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643000cd80 con 0x7f643c101890 2024-04-24T23:11:00.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c19f870 con 0x7f643c101890 2024-04-24T23:11:00.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f6438ff9640 1 -- 172.21.15.102:0/3421871274 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64300108e0 con 0x7f643c101890 2024-04-24T23:11:00.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f6438ff9640 1 -- 172.21.15.102:0/3421871274 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430018920 con 0x7f643c101890 2024-04-24T23:11:00.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.420+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c19fdc0 con 0x7f643c101890 2024-04-24T23:11:00.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.421+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c1079c0 con 0x7f643c101890 2024-04-24T23:11:00.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.422+0000 7f6438ff9640 1 -- 172.21.15.102:0/3421871274 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6430010a90 con 0x7f643c101890 2024-04-24T23:11:00.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.422+0000 7f6438ff9640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64240a2b90 0x7f64240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:00.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.423+0000 7f643b7fe640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64240a2b90 0x7f64240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:00.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.423+0000 7f6438ff9640 1 -- 172.21.15.102:0/3421871274 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f64300cf640 con 0x7f643c101890 2024-04-24T23:11:00.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.423+0000 7f643b7fe640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64240a2b90 0x7f64240a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f643c1016f0 tx=0x7f642c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:00.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.426+0000 7f6438ff9640 1 -- 172.21.15.102:0/3421871274 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f643009c100 con 0x7f643c101890 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1684727205' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1153735689' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:00.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1684727205' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:11:00.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:11:00.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1153735689' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:11:00.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:00.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:00.642+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f643c0623b0 con 0x7f643c101890 2024-04-24T23:11:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1684727205' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:11:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:11:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1153735689' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:11:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:00.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:00.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:00.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:11:01.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.290+0000 7f6438ff9640 1 -- 172.21.15.102:0/3421871274 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f64300a0400 con 0x7f643c101890 2024-04-24T23:11:01.291 INFO:teuthology.orchestra.run.smithi102.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64240a2b90 msgr2=0x7f64240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64240a2b90 0x7f64240a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f643c1016f0 tx=0x7f642c005f90 comp rx=0 tx=0).stop 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101890 msgr2=0x7f643c197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101890 0x7f643c197ea0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f643000e9f0 tx=0x7f643000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 shutdown_connections 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64240a2b90 0x7f64240a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c102250 0x7f643c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101890 0x7f643c197ea0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 --2- 172.21.15.102:0/3421871274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643c1006a0 0x7f643c197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 >> 172.21.15.102:0/3421871274 conn(0x7f643c0fbc30 msgr2=0x7f643c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 shutdown_connections 2024-04-24T23:11:01.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.293+0000 7f64421d5640 1 -- 172.21.15.102:0/3421871274 wait complete. 2024-04-24T23:11:01.309 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 -- 172.21.15.102:0/3512496770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80100690 msgr2=0x7f1e80100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/3512496770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80100690 0x7f1e80100ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009900 tx=0x7f1e7802a410 comp rx=0 tx=0).stop 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 -- 172.21.15.102:0/3512496770 shutdown_connections 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/3512496770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e80102240 0x7f1e80108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/3512496770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e80101880 0x7f1e80101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/3512496770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80100690 0x7f1e80100ab0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 -- 172.21.15.102:0/3512496770 >> 172.21.15.102:0/3512496770 conn(0x7f1e800fbc40 msgr2=0x7f1e800fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 -- 172.21.15.102:0/3512496770 shutdown_connections 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 -- 172.21.15.102:0/3512496770 wait complete. 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 Processor -- start 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 -- start start 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e80100690 0x7f1e80199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:01.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e80101880 0x7f1e8019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80102240 0x7f1e801a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8010c290 con 0x7f1e80102240 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8010c110 con 0x7f1e80101880 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.409+0000 7f1e87ff9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8010c410 con 0x7f1e80100690 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e86596640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80102240 0x7f1e801a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e86596640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80102240 0x7f1e801a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40332/0 (socket says 172.21.15.102:40332) 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e86596640 1 -- 172.21.15.102:0/2728987195 learned_addr learned my addr 172.21.15.102:0/2728987195 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e85594640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e80101880 0x7f1e8019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e85d95640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e80100690 0x7f1e80199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e86596640 1 -- 172.21.15.102:0/2728987195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e80100690 msgr2=0x7f1e80199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e86596640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e80100690 0x7f1e80199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e86596640 1 -- 172.21.15.102:0/2728987195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e80101880 msgr2=0x7f1e8019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e86596640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e80101880 0x7f1e8019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e86596640 1 -- 172.21.15.102:0/2728987195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e801a1970 con 0x7f1e80102240 2024-04-24T23:11:01.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e86596640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80102240 0x7f1e801a1300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009fd0 tx=0x7f1e7000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:01.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e7effd640 1 -- 172.21.15.102:0/2728987195 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70011830 con 0x7f1e80102240 2024-04-24T23:11:01.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e801a1c60 con 0x7f1e80102240 2024-04-24T23:11:01.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e7effd640 1 -- 172.21.15.102:0/2728987195 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e70016070 con 0x7f1e80102240 2024-04-24T23:11:01.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e7effd640 1 -- 172.21.15.102:0/2728987195 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70010de0 con 0x7f1e80102240 2024-04-24T23:11:01.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.410+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e801a2160 con 0x7f1e80102240 2024-04-24T23:11:01.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.411+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e801a23f0 con 0x7f1e80102240 2024-04-24T23:11:01.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.412+0000 7f1e7effd640 1 -- 172.21.15.102:0/2728987195 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1e700119d0 con 0x7f1e80102240 2024-04-24T23:11:01.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.413+0000 7f1e7effd640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1e680a2b90 0x7f1e680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:01.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.413+0000 7f1e85d95640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1e680a2b90 0x7f1e680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:01.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.413+0000 7f1e7effd640 1 -- 172.21.15.102:0/2728987195 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1e700cece0 con 0x7f1e80102240 2024-04-24T23:11:01.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.413+0000 7f1e85d95640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1e680a2b90 0x7f1e680a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1e7802a930 tx=0x7f1e78000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:01.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.416+0000 7f1e7effd640 1 -- 172.21.15.102:0/2728987195 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1e7009bd90 con 0x7f1e80102240 2024-04-24T23:11:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:01 smithi102 ceph-mon[29216]: pgmap v289: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:01 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1153735689' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:11:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:01 smithi102 ceph-mon[29216]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:11:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:01 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3421871274' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:11:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:01 smithi102 ceph-mon[29216]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T23:11:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:01 smithi102 ceph-mon[29216]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:11:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:01 smithi102 ceph-mon[29216]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:11:01.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.601+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e801a2660 con 0x7f1e80102240 2024-04-24T23:11:01.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.602+0000 7f1e7effd640 1 -- 172.21.15.102:0/2728987195 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22292 (secure 0 0 0) 0x7f1e7009b690 con 0x7f1e80102240 2024-04-24T23:11:01.604 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:11:01.604 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":85,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:11:00.642746+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1e680a2b90 msgr2=0x7f1e680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1e680a2b90 0x7f1e680a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1e7802a930 tx=0x7f1e78000bc0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80102240 msgr2=0x7f1e801a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80102240 0x7f1e801a1300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009fd0 tx=0x7f1e7000b8e0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 shutdown_connections 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1e680a2b90 0x7f1e680a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e80102240 0x7f1e801a1300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e80101880 0x7f1e8019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 --2- 172.21.15.102:0/2728987195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e80100690 0x7f1e80199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.604+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 >> 172.21.15.102:0/2728987195 conn(0x7f1e800fbc40 msgr2=0x7f1e80069e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.605+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 shutdown_connections 2024-04-24T23:11:01.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.605+0000 7f1e87ff9640 1 -- 172.21.15.102:0/2728987195 wait complete. 2024-04-24T23:11:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:01 smithi161 ceph-mon[32709]: pgmap v289: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:01 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1153735689' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:11:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:01 smithi161 ceph-mon[32709]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:11:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:01 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3421871274' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:11:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:01 smithi161 ceph-mon[32709]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T23:11:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:01 smithi161 ceph-mon[32709]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:11:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:01 smithi161 ceph-mon[32709]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:11:01.617 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.714+0000 7fe96481c640 1 -- 172.21.15.102:0/1967584173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9601033f0 msgr2=0x7fe960109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.714+0000 7fe96481c640 1 --2- 172.21.15.102:0/1967584173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9601033f0 0x7fe960109c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe958009cd0 tx=0x7fe95802a8e0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.714+0000 7fe96481c640 1 -- 172.21.15.102:0/1967584173 shutdown_connections 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.714+0000 7fe96481c640 1 --2- 172.21.15.102:0/1967584173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9601033f0 0x7fe960109c80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.714+0000 7fe96481c640 1 --2- 172.21.15.102:0/1967584173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe960102a30 0x7fe960102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.714+0000 7fe96481c640 1 --2- 172.21.15.102:0/1967584173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9600ffa60 0x7fe9600ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.714+0000 7fe96481c640 1 -- 172.21.15.102:0/1967584173 >> 172.21.15.102:0/1967584173 conn(0x7fe9600fb3b0 msgr2=0x7fe9600fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.714+0000 7fe96481c640 1 -- 172.21.15.102:0/1967584173 shutdown_connections 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.714+0000 7fe96481c640 1 -- 172.21.15.102:0/1967584173 wait complete. 2024-04-24T23:11:01.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe96481c640 1 Processor -- start 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe96481c640 1 -- start start 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe96481c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9600ffa60 0x7fe9601979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe96481c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe960102a30 0x7fe960197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe96481c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9601033f0 0x7fe96019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe96481c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010d2f0 con 0x7fe960102a30 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe96481c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010d170 con 0x7fe9600ffa60 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe96481c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010d470 con 0x7fe9601033f0 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe95dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe960102a30 0x7fe960197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe95dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe960102a30 0x7fe960197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40352/0 (socket says 172.21.15.102:40352) 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.715+0000 7fe95dd9b640 1 -- 172.21.15.102:0/1064608534 learned_addr learned my addr 172.21.15.102:0/1064608534 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe95e59c640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9600ffa60 0x7fe9601979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe95ed9d640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9601033f0 0x7fe96019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe95dd9b640 1 -- 172.21.15.102:0/1064608534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9601033f0 msgr2=0x7fe96019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe95dd9b640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9601033f0 0x7fe96019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe95dd9b640 1 -- 172.21.15.102:0/1064608534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9600ffa60 msgr2=0x7fe9601979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe95dd9b640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9600ffa60 0x7fe9601979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe95dd9b640 1 -- 172.21.15.102:0/1064608534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96019f5c0 con 0x7fe960102a30 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe95dd9b640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe960102a30 0x7fe960197ee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe95400d770 tx=0x7fe95400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe94f7fe640 1 -- 172.21.15.102:0/1064608534 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954017070 con 0x7fe960102a30 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe94f7fe640 1 -- 172.21.15.102:0/1064608534 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9540054d0 con 0x7fe960102a30 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96019f8b0 con 0x7fe960102a30 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe94f7fe640 1 -- 172.21.15.102:0/1064608534 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954004cf0 con 0x7fe960102a30 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.716+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe96019fdb0 con 0x7fe960102a30 2024-04-24T23:11:01.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.717+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe960108b60 con 0x7fe960102a30 2024-04-24T23:11:01.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.718+0000 7fe94f7fe640 1 -- 172.21.15.102:0/1064608534 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe954004e90 con 0x7fe960102a30 2024-04-24T23:11:01.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.719+0000 7fe94f7fe640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9480a2b90 0x7fe9480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:01.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.719+0000 7fe95e59c640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9480a2b90 0x7fe9480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:01.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.719+0000 7fe94f7fe640 1 -- 172.21.15.102:0/1064608534 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe9540cee60 con 0x7fe960102a30 2024-04-24T23:11:01.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.720+0000 7fe95e59c640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9480a2b90 0x7fe9480a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe950005db0 tx=0x7fe950005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:01.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.723+0000 7fe94f7fe640 1 -- 172.21.15.102:0/1064608534 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe95409bf10 con 0x7fe960102a30 2024-04-24T23:11:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:01 smithi183 ceph-mon[33137]: pgmap v289: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:01 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1153735689' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:11:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:01 smithi183 ceph-mon[33137]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:11:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:01 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3421871274' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:11:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:01 smithi183 ceph-mon[33137]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T23:11:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:01 smithi183 ceph-mon[33137]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:11:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:01 smithi183 ceph-mon[33137]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:11:01.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.898+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe96002d290 con 0x7fe960102a30 2024-04-24T23:11:01.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.900+0000 7fe94f7fe640 1 -- 172.21.15.102:0/1064608534 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22292 (secure 0 0 0) 0x7fe95409b810 con 0x7fe960102a30 2024-04-24T23:11:01.901 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:11:01.901 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":85,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:11:00.642746+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9480a2b90 msgr2=0x7fe9480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9480a2b90 0x7fe9480a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe950005db0 tx=0x7fe950005d40 comp rx=0 tx=0).stop 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe960102a30 msgr2=0x7fe960197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe960102a30 0x7fe960197ee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe95400d770 tx=0x7fe95400dc30 comp rx=0 tx=0).stop 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 shutdown_connections 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9480a2b90 0x7fe9480a5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9601033f0 0x7fe96019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe960102a30 0x7fe960197ee0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 --2- 172.21.15.102:0/1064608534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9600ffa60 0x7fe9601979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:01.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 >> 172.21.15.102:0/1064608534 conn(0x7fe9600fb3b0 msgr2=0x7fe960105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:01.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 shutdown_connections 2024-04-24T23:11:01.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:01.902+0000 7fe96481c640 1 -- 172.21.15.102:0/1064608534 wait complete. 2024-04-24T23:11:01.914 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:11:02.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.014+0000 7fb1f577e640 1 -- 172.21.15.102:0/1364046509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 msgr2=0x7fb1f0106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.014+0000 7fb1f577e640 1 --2- 172.21.15.102:0/1364046509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 0x7fb1f0106f20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009900 tx=0x7fb1e002a410 comp rx=0 tx=0).stop 2024-04-24T23:11:02.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.014+0000 7fb1f577e640 1 -- 172.21.15.102:0/1364046509 shutdown_connections 2024-04-24T23:11:02.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.014+0000 7fb1f577e640 1 --2- 172.21.15.102:0/1364046509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f0107460 0x7fb1f01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.014+0000 7fb1f577e640 1 --2- 172.21.15.102:0/1364046509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 0x7fb1f0106f20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.014+0000 7fb1f577e640 1 --2- 172.21.15.102:0/1364046509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1f0102200 0x7fb1f01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.014+0000 7fb1f577e640 1 -- 172.21.15.102:0/1364046509 >> 172.21.15.102:0/1364046509 conn(0x7fb1f00fbbf0 msgr2=0x7fb1f00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.014+0000 7fb1f577e640 1 -- 172.21.15.102:0/1364046509 shutdown_connections 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.014+0000 7fb1f577e640 1 -- 172.21.15.102:0/1364046509 wait complete. 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1f577e640 1 Processor -- start 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1f577e640 1 -- start start 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1f577e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f0102200 0x7fb1f0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1f577e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 0x7fb1f0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1f577e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1f0107460 0x7fb1f019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1f577e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f010d310 con 0x7fb1f0104b30 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1f577e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f010d190 con 0x7fb1f0102200 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1f577e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f010d490 con 0x7fb1f0107460 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1ee7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 0x7fb1f0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1ee7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 0x7fb1f0198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40360/0 (socket says 172.21.15.102:40360) 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1ee7fc640 1 -- 172.21.15.102:0/737619364 learned_addr learned my addr 172.21.15.102:0/737619364 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.015+0000 7fb1ef7fe640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1f0107460 0x7fb1f019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1eeffd640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f0102200 0x7fb1f0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1ee7fc640 1 -- 172.21.15.102:0/737619364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1f0107460 msgr2=0x7fb1f019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1ee7fc640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1f0107460 0x7fb1f019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1ee7fc640 1 -- 172.21.15.102:0/737619364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f0102200 msgr2=0x7fb1f0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1ee7fc640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f0102200 0x7fb1f0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1ee7fc640 1 -- 172.21.15.102:0/737619364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f019f710 con 0x7fb1f0104b30 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1eeffd640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f0102200 0x7fb1f0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1ee7fc640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 0x7fb1f0198030 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00314e0 tx=0x7fb1e00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1d7fff640 1 -- 172.21.15.102:0/737619364 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e003f030 con 0x7fb1f0104b30 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1d7fff640 1 -- 172.21.15.102:0/737619364 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1e0003cb0 con 0x7fb1f0104b30 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f019f9a0 con 0x7fb1f0104b30 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1d7fff640 1 -- 172.21.15.102:0/737619364 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0035070 con 0x7fb1f0104b30 2024-04-24T23:11:02.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.016+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f019fe90 con 0x7fb1f0104b30 2024-04-24T23:11:02.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.017+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f00fd6e0 con 0x7fb1f0104b30 2024-04-24T23:11:02.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.018+0000 7fb1d7fff640 1 -- 172.21.15.102:0/737619364 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb1e0042030 con 0x7fb1f0104b30 2024-04-24T23:11:02.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.018+0000 7fb1d7fff640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb1d80a2b90 0x7fb1d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.019+0000 7fb1eeffd640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb1d80a2b90 0x7fb1d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.019+0000 7fb1d7fff640 1 -- 172.21.15.102:0/737619364 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb1e00ef9e0 con 0x7fb1f0104b30 2024-04-24T23:11:02.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.019+0000 7fb1eeffd640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb1d80a2b90 0x7fb1d80a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006fd0 tx=0x7fb1e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:02.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.022+0000 7fb1d7fff640 1 -- 172.21.15.102:0/737619364 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1e00bca90 con 0x7fb1f0104b30 2024-04-24T23:11:02.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.238+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb1f002d170 con 0x7fb1f0104b30 2024-04-24T23:11:02.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.308+0000 7fb1d7fff640 1 -- 172.21.15.102:0/737619364 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fb1e00bc390 con 0x7fb1f0104b30 2024-04-24T23:11:02.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb1d80a2b90 msgr2=0x7fb1d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb1d80a2b90 0x7fb1d80a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006fd0 tx=0x7fb1e4008040 comp rx=0 tx=0).stop 2024-04-24T23:11:02.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 msgr2=0x7fb1f0198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 0x7fb1f0198030 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00314e0 tx=0x7fb1e00042e0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 shutdown_connections 2024-04-24T23:11:02.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb1d80a2b90 0x7fb1d80a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1f0107460 0x7fb1f019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1f0104b30 0x7fb1f0198030 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 --2- 172.21.15.102:0/737619364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f0102200 0x7fb1f0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 >> 172.21.15.102:0/737619364 conn(0x7fb1f00fbbf0 msgr2=0x7fb1f0102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:02.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 shutdown_connections 2024-04-24T23:11:02.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.310+0000 7fb1f577e640 1 -- 172.21.15.102:0/737619364 wait complete. 2024-04-24T23:11:02.322 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-24T23:11:02.323 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T23:11:02.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.422+0000 7f1dab57e640 1 -- 172.21.15.102:0/1720906557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 msgr2=0x7f1da4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.422+0000 7f1dab57e640 1 --2- 172.21.15.102:0/1720906557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 0x7f1da4108ac0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009900 tx=0x7f1da002a410 comp rx=0 tx=0).stop 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.422+0000 7f1dab57e640 1 -- 172.21.15.102:0/1720906557 shutdown_connections 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.422+0000 7f1dab57e640 1 --2- 172.21.15.102:0/1720906557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 0x7f1da4108ac0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.422+0000 7f1dab57e640 1 --2- 172.21.15.102:0/1720906557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1da4101870 0x7f1da4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.422+0000 7f1dab57e640 1 --2- 172.21.15.102:0/1720906557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1da4100680 0x7f1da4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.422+0000 7f1dab57e640 1 -- 172.21.15.102:0/1720906557 >> 172.21.15.102:0/1720906557 conn(0x7f1da40fbbf0 msgr2=0x7f1da40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 -- 172.21.15.102:0/1720906557 shutdown_connections 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 -- 172.21.15.102:0/1720906557 wait complete. 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 Processor -- start 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 -- start start 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1da4100680 0x7f1da4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1da4101870 0x7f1da4197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 0x7f1da419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410c210 con 0x7f1da4102230 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410c090 con 0x7f1da4101870 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1dab57e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410c390 con 0x7f1da4100680 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1da9b1b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 0x7f1da419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1da9b1b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 0x7f1da419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40380/0 (socket says 172.21.15.102:40380) 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1da9b1b640 1 -- 172.21.15.102:0/4109160778 learned_addr learned my addr 172.21.15.102:0/4109160778 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.423+0000 7f1da931a640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1da4100680 0x7f1da4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1da8b19640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1da4101870 0x7f1da4197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1da9b1b640 1 -- 172.21.15.102:0/4109160778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1da4100680 msgr2=0x7f1da4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1da9b1b640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1da4100680 0x7f1da4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1da9b1b640 1 -- 172.21.15.102:0/4109160778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1da4101870 msgr2=0x7f1da4197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1da9b1b640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1da4101870 0x7f1da4197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1da9b1b640 1 -- 172.21.15.102:0/4109160778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da419f650 con 0x7f1da4102230 2024-04-24T23:11:02.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1da8b19640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1da4101870 0x7f1da4197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:02.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1da9b1b640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 0x7f1da419efe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005940 tx=0x7f1da00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:02.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1d9a7fc640 1 -- 172.21.15.102:0/4109160778 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da003f030 con 0x7f1da4102230 2024-04-24T23:11:02.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1d9a7fc640 1 -- 172.21.15.102:0/4109160778 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1da0003cb0 con 0x7f1da4102230 2024-04-24T23:11:02.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da419f8e0 con 0x7f1da4102230 2024-04-24T23:11:02.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1d9a7fc640 1 -- 172.21.15.102:0/4109160778 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0035070 con 0x7f1da4102230 2024-04-24T23:11:02.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.424+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da419fea0 con 0x7f1da4102230 2024-04-24T23:11:02.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.425+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da41a00d0 con 0x7f1da4102230 2024-04-24T23:11:02.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.426+0000 7f1d9a7fc640 1 -- 172.21.15.102:0/4109160778 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1da0042030 con 0x7f1da4102230 2024-04-24T23:11:02.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.426+0000 7f1d9a7fc640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d900a2c10 0x7f1d900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.427+0000 7f1da931a640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d900a2c10 0x7f1d900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.427+0000 7f1d9a7fc640 1 -- 172.21.15.102:0/4109160778 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1da00ef550 con 0x7f1da4102230 2024-04-24T23:11:02.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.427+0000 7f1da931a640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d900a2c10 0x7f1d900a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1da41016d0 tx=0x7f1d9c00a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:02.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.431+0000 7f1d9a7fc640 1 -- 172.21.15.102:0/4109160778 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1da00bc600 con 0x7f1da4102230 2024-04-24T23:11:02.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:02 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3421871274' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:11:02.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:02 smithi102 ceph-mon[29216]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T23:11:02.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:02 smithi102 ceph-mon[29216]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T23:11:02.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:02 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2728987195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:02.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:02 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1064608534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:02.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:02 smithi102 ceph-mon[29216]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:11:02.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:02 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/737619364' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:11:02.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:02 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3421871274' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:11:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:02 smithi161 ceph-mon[32709]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T23:11:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:02 smithi161 ceph-mon[32709]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T23:11:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:02 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2728987195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:02 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1064608534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:02 smithi161 ceph-mon[32709]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:11:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:02 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/737619364' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:11:02.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:02.646+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f1da41a0460 con 0x7f1da4102230 2024-04-24T23:11:02.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:02 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3421871274' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:11:02.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:02 smithi183 ceph-mon[33137]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T23:11:02.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:02 smithi183 ceph-mon[33137]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T23:11:02.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:02 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2728987195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:02.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:02 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1064608534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:02.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:02 smithi183 ceph-mon[33137]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:11:02.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:02 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/737619364' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:11:03.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.315+0000 7f1d9a7fc640 1 -- 172.21.15.102:0/4109160778 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f1da00bbf00 con 0x7f1da4102230 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.317+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d900a2c10 msgr2=0x7f1d900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.317+0000 7f1dab57e640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d900a2c10 0x7f1d900a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1da41016d0 tx=0x7f1d9c00a590 comp rx=0 tx=0).stop 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.317+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 msgr2=0x7f1da419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.317+0000 7f1dab57e640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 0x7f1da419efe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005940 tx=0x7f1da00043c0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.318+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 shutdown_connections 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.318+0000 7f1dab57e640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d900a2c10 0x7f1d900a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.318+0000 7f1dab57e640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da4102230 0x7f1da419efe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.318+0000 7f1dab57e640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1da4101870 0x7f1da4197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.318+0000 7f1dab57e640 1 --2- 172.21.15.102:0/4109160778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1da4100680 0x7f1da4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.318+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 >> 172.21.15.102:0/4109160778 conn(0x7f1da40fbbf0 msgr2=0x7f1da40688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.318+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 shutdown_connections 2024-04-24T23:11:03.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.318+0000 7f1dab57e640 1 -- 172.21.15.102:0/4109160778 wait complete. 2024-04-24T23:11:03.333 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-24T23:11:03.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.436+0000 7fb43049e640 1 -- 172.21.15.102:0/2343536492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 msgr2=0x7fb428102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.436+0000 7fb43049e640 1 --2- 172.21.15.102:0/2343536492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 0x7fb428102e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb418009900 tx=0x7fb41802a410 comp rx=0 tx=0).stop 2024-04-24T23:11:03.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 -- 172.21.15.102:0/2343536492 shutdown_connections 2024-04-24T23:11:03.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 --2- 172.21.15.102:0/2343536492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb428103350 0x7fb428109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 --2- 172.21.15.102:0/2343536492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 0x7fb428102e10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 --2- 172.21.15.102:0/2343536492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4281017a0 0x7fb428101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 -- 172.21.15.102:0/2343536492 >> 172.21.15.102:0/2343536492 conn(0x7fb4280fcd10 msgr2=0x7fb4280ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 -- 172.21.15.102:0/2343536492 shutdown_connections 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 -- 172.21.15.102:0/2343536492 wait complete. 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 Processor -- start 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 -- start start 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4281017a0 0x7fb428197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 0x7fb428198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb428103350 0x7fb42819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb42810d3c0 con 0x7fb428102990 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb42810d240 con 0x7fb428103350 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb43049e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb42810d540 con 0x7fb4281017a0 2024-04-24T23:11:03.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb42da39640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 0x7fb428198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb42da39640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 0x7fb428198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40400/0 (socket says 172.21.15.102:40400) 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.437+0000 7fb42da39640 1 -- 172.21.15.102:0/3344544063 learned_addr learned my addr 172.21.15.102:0/3344544063 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb42e23a640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4281017a0 0x7fb428197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb42da39640 1 -- 172.21.15.102:0/3344544063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4281017a0 msgr2=0x7fb428197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb42da39640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4281017a0 0x7fb428197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb42da39640 1 -- 172.21.15.102:0/3344544063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb428103350 msgr2=0x7fb42819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb42da39640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb428103350 0x7fb42819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb42da39640 1 -- 172.21.15.102:0/3344544063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42819f770 con 0x7fb428102990 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb42e23a640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4281017a0 0x7fb428197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb42da39640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 0x7fb428198090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb418005940 tx=0x7fb418004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb41f7fe640 1 -- 172.21.15.102:0/3344544063 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41803f030 con 0x7fb428102990 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb41f7fe640 1 -- 172.21.15.102:0/3344544063 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb418003cb0 con 0x7fb428102990 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42819fa00 con 0x7fb428102990 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb41f7fe640 1 -- 172.21.15.102:0/3344544063 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418035070 con 0x7fb428102990 2024-04-24T23:11:03.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.438+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42819ff70 con 0x7fb428102990 2024-04-24T23:11:03.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.439+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb428108b80 con 0x7fb428102990 2024-04-24T23:11:03.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.440+0000 7fb41f7fe640 1 -- 172.21.15.102:0/3344544063 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb418042030 con 0x7fb428102990 2024-04-24T23:11:03.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.440+0000 7fb41f7fe640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4140a2b90 0x7fb4140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:03.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.441+0000 7fb42e23a640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4140a2b90 0x7fb4140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:03.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.441+0000 7fb41f7fe640 1 -- 172.21.15.102:0/3344544063 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb4180efa40 con 0x7fb428102990 2024-04-24T23:11:03.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.441+0000 7fb42e23a640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4140a2b90 0x7fb4140a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:03.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.445+0000 7fb41f7fe640 1 -- 172.21.15.102:0/3344544063 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb4180bcaf0 con 0x7fb428102990 2024-04-24T23:11:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:03 smithi102 ceph-mon[29216]: pgmap v293: 97 pgs: 12 creating+peering, 26 active+clean, 59 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-24T23:11:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:03 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/737619364' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:11:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:03 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] up:active 2024-04-24T23:11:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:03 smithi102 ceph-mon[29216]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:11:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:03 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4109160778' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T23:11:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:03 smithi102 ceph-mon[29216]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:11:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:03 smithi102 ceph-mon[29216]: Cluster is now healthy 2024-04-24T23:11:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:03 smithi161 ceph-mon[32709]: pgmap v293: 97 pgs: 12 creating+peering, 26 active+clean, 59 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-24T23:11:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:03 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/737619364' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:11:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:03 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] up:active 2024-04-24T23:11:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:03 smithi161 ceph-mon[32709]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:11:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:03 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4109160778' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T23:11:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:03 smithi161 ceph-mon[32709]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:11:03.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:03 smithi161 ceph-mon[32709]: Cluster is now healthy 2024-04-24T23:11:03.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:03.659+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb42802d3c0 con 0x7fb428102990 2024-04-24T23:11:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:03 smithi183 ceph-mon[33137]: pgmap v293: 97 pgs: 12 creating+peering, 26 active+clean, 59 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-24T23:11:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:03 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/737619364' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:11:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:03 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] up:active 2024-04-24T23:11:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:03 smithi183 ceph-mon[33137]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:11:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:03 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4109160778' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T23:11:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:03 smithi183 ceph-mon[33137]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:11:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:03 smithi183 ceph-mon[33137]: Cluster is now healthy 2024-04-24T23:11:04.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.326+0000 7fb41f7fe640 1 -- 172.21.15.102:0/3344544063 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fb4180bc3f0 con 0x7fb428102990 2024-04-24T23:11:04.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4140a2b90 msgr2=0x7fb4140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4140a2b90 0x7fb4140a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424008040 comp rx=0 tx=0).stop 2024-04-24T23:11:04.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 msgr2=0x7fb428198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 0x7fb428198090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb418005940 tx=0x7fb418004430 comp rx=0 tx=0).stop 2024-04-24T23:11:04.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 shutdown_connections 2024-04-24T23:11:04.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4140a2b90 0x7fb4140a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb428103350 0x7fb42819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb428102990 0x7fb428198090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 --2- 172.21.15.102:0/3344544063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4281017a0 0x7fb428197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 >> 172.21.15.102:0/3344544063 conn(0x7fb4280fcd10 msgr2=0x7fb428105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:04.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 shutdown_connections 2024-04-24T23:11:04.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.329+0000 7fb43049e640 1 -- 172.21.15.102:0/3344544063 wait complete. 2024-04-24T23:11:04.345 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T23:11:04.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 -- 172.21.15.102:0/1314061664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280107480 msgr2=0x7f22801098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 --2- 172.21.15.102:0/1314061664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280107480 0x7f22801098d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2278009ca0 tx=0x7f227802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 -- 172.21.15.102:0/1314061664 shutdown_connections 2024-04-24T23:11:04.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 --2- 172.21.15.102:0/1314061664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280107480 0x7f22801098d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 --2- 172.21.15.102:0/1314061664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2280104b50 0x7f2280106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 --2- 172.21.15.102:0/1314061664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2280102220 0x7f2280104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 -- 172.21.15.102:0/1314061664 >> 172.21.15.102:0/1314061664 conn(0x7f22800fbc10 msgr2=0x7f22800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 -- 172.21.15.102:0/1314061664 shutdown_connections 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 -- 172.21.15.102:0/1314061664 wait complete. 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 Processor -- start 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.447+0000 7f228723f640 1 -- start start 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f228723f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2280102220 0x7f2280197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f228723f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280104b50 0x7f2280197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f228723f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2280107480 0x7f228019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f228723f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f228010d340 con 0x7f2280104b50 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f228723f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f228010d1c0 con 0x7f2280102220 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f228723f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f228010d4c0 con 0x7f2280107480 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280104b50 0x7f2280197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:04.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280104b50 0x7f2280197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40428/0 (socket says 172.21.15.102:40428) 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227ffff640 1 -- 172.21.15.102:0/2838790241 learned_addr learned my addr 172.21.15.102:0/2838790241 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f2284fdb640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2280102220 0x7f2280197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f22857dc640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2280107480 0x7f228019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227ffff640 1 -- 172.21.15.102:0/2838790241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2280107480 msgr2=0x7f228019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227ffff640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2280107480 0x7f228019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227ffff640 1 -- 172.21.15.102:0/2838790241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2280102220 msgr2=0x7f2280197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227ffff640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2280102220 0x7f2280197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227ffff640 1 -- 172.21.15.102:0/2838790241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228019f630 con 0x7f2280104b50 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f2284fdb640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2280102220 0x7f2280197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f22857dc640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2280107480 0x7f228019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227ffff640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280104b50 0x7f2280197f50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f226c00b450 tx=0x7f226c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227dffb640 1 -- 172.21.15.102:0/2838790241 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c015020 con 0x7f2280104b50 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227dffb640 1 -- 172.21.15.102:0/2838790241 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f226c011d40 con 0x7f2280104b50 2024-04-24T23:11:04.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228019f920 con 0x7f2280104b50 2024-04-24T23:11:04.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.448+0000 7f227dffb640 1 -- 172.21.15.102:0/2838790241 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c00f340 con 0x7f2280104b50 2024-04-24T23:11:04.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.449+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228019fe70 con 0x7f2280104b50 2024-04-24T23:11:04.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.449+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22800fd700 con 0x7f2280104b50 2024-04-24T23:11:04.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.450+0000 7f227dffb640 1 -- 172.21.15.102:0/2838790241 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f226c018020 con 0x7f2280104b50 2024-04-24T23:11:04.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.451+0000 7f227dffb640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f22700a2b90 0x7f22700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:04.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.451+0000 7f2284fdb640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f22700a2b90 0x7f22700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:04.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.451+0000 7f227dffb640 1 -- 172.21.15.102:0/2838790241 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f226c0ce9c0 con 0x7f2280104b50 2024-04-24T23:11:04.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.451+0000 7f2284fdb640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f22700a2b90 0x7f22700a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2274009fc0 tx=0x7f2274005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:04.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.456+0000 7f227dffb640 1 -- 172.21.15.102:0/2838790241 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f226c0d0e20 con 0x7f2280104b50 2024-04-24T23:11:04.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:04 smithi102 ceph-mon[29216]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:11:04.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:04 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4109160778' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T23:11:04.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:04 smithi102 ceph-mon[29216]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:11:04.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:04 smithi102 ceph-mon[29216]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:11:04.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:04 smithi102 ceph-mon[29216]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-24T23:11:04.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:04 smithi102 ceph-mon[29216]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-24T23:11:04.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:04 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3344544063' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:11:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:04 smithi161 ceph-mon[32709]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:11:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:04 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4109160778' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T23:11:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:04 smithi161 ceph-mon[32709]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:11:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:04 smithi161 ceph-mon[32709]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:11:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:04 smithi161 ceph-mon[32709]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-24T23:11:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:04 smithi161 ceph-mon[32709]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-24T23:11:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:04 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3344544063' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:11:04.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.671+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f228002d3c0 con 0x7f2280104b50 2024-04-24T23:11:04.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:04 smithi183 ceph-mon[33137]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:11:04.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:04 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4109160778' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T23:11:04.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:04 smithi183 ceph-mon[33137]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:11:04.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:04 smithi183 ceph-mon[33137]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:11:04.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:04 smithi183 ceph-mon[33137]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-24T23:11:04.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:04 smithi183 ceph-mon[33137]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-24T23:11:04.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:04 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3344544063' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:11:04.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.859+0000 7f227dffb640 1 -- 172.21.15.102:0/2838790241 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f226c01f020 con 0x7f2280104b50 2024-04-24T23:11:04.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f22700a2b90 msgr2=0x7f22700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f22700a2b90 0x7f22700a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2274009fc0 tx=0x7f2274005f90 comp rx=0 tx=0).stop 2024-04-24T23:11:04.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280104b50 msgr2=0x7f2280197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280104b50 0x7f2280197f50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f226c00b450 tx=0x7f226c00b910 comp rx=0 tx=0).stop 2024-04-24T23:11:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 shutdown_connections 2024-04-24T23:11:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f22700a2b90 0x7f22700a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2280107480 0x7f228019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2280104b50 0x7f2280197f50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 --2- 172.21.15.102:0/2838790241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2280102220 0x7f2280197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 >> 172.21.15.102:0/2838790241 conn(0x7f22800fbc10 msgr2=0x7f2280102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 shutdown_connections 2024-04-24T23:11:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.861+0000 7f228723f640 1 -- 172.21.15.102:0/2838790241 wait complete. 2024-04-24T23:11:04.874 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-24T23:11:04.875 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-24T23:11:04.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2961886266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 msgr2=0x7fbe28109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2961886266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 0x7fbe28109bd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009ca0 tx=0x7fbe2002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2961886266 shutdown_connections 2024-04-24T23:11:04.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2961886266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 0x7fbe28109bd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2961886266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe28102980 0x7fbe28102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2961886266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe28101790 0x7fbe28101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2961886266 >> 172.21.15.102:0/2961886266 conn(0x7fbe280fcd20 msgr2=0x7fbe280ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:04.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2961886266 shutdown_connections 2024-04-24T23:11:04.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2961886266 wait complete. 2024-04-24T23:11:04.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 Processor -- start 2024-04-24T23:11:04.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.975+0000 7fbe2e15c640 1 -- start start 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe2e15c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe28101790 0x7fbe281979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe2e15c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe28102980 0x7fbe28197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe2e15c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 0x7fbe2819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe2e15c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2810d320 con 0x7fbe28103340 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe2e15c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2810d1a0 con 0x7fbe28101790 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe2e15c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2810d4a0 con 0x7fbe28102980 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe27fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 0x7fbe2819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe27fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 0x7fbe2819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40440/0 (socket says 172.21.15.102:40440) 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe27fff640 1 -- 172.21.15.102:0/2178225619 learned_addr learned my addr 172.21.15.102:0/2178225619 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe277fe640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe28101790 0x7fbe281979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe27fff640 1 -- 172.21.15.102:0/2178225619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe28102980 msgr2=0x7fbe28197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe1effd640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe28102980 0x7fbe28197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe27fff640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe28102980 0x7fbe28197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe27fff640 1 -- 172.21.15.102:0/2178225619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe28101790 msgr2=0x7fbe281979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:04.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe27fff640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe28101790 0x7fbe281979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:04.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe27fff640 1 -- 172.21.15.102:0/2178225619 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2819f5f0 con 0x7fbe28103340 2024-04-24T23:11:04.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe27fff640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 0x7fbe2819ef80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009dc0 tx=0x7fbe20003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:04.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe257fa640 1 -- 172.21.15.102:0/2178225619 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20032ec0 con 0x7fbe28103340 2024-04-24T23:11:04.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe257fa640 1 -- 172.21.15.102:0/2178225619 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe200044f0 con 0x7fbe28103340 2024-04-24T23:11:04.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2819f880 con 0x7fbe28103340 2024-04-24T23:11:04.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe257fa640 1 -- 172.21.15.102:0/2178225619 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20030d40 con 0x7fbe28103340 2024-04-24T23:11:04.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.976+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2819fdd0 con 0x7fbe28103340 2024-04-24T23:11:04.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.977+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe28108ab0 con 0x7fbe28103340 2024-04-24T23:11:04.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.978+0000 7fbe257fa640 1 -- 172.21.15.102:0/2178225619 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbe20004040 con 0x7fbe28103340 2024-04-24T23:11:04.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.979+0000 7fbe257fa640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbe140a2d30 0x7fbe140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:04.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.979+0000 7fbe277fe640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbe140a2d30 0x7fbe140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:04.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.979+0000 7fbe257fa640 1 -- 172.21.15.102:0/2178225619 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbe200f0720 con 0x7fbe28103340 2024-04-24T23:11:04.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.979+0000 7fbe277fe640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbe140a2d30 0x7fbe140a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbe281027e0 tx=0x7fbe1800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:04.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:04.982+0000 7fbe257fa640 1 -- 172.21.15.102:0/2178225619 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbe200bd7d0 con 0x7fbe28103340 2024-04-24T23:11:05.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.185+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbe28061b80 con 0x7fbe140a2d30 2024-04-24T23:11:05.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.211+0000 7fbe257fa640 1 -- 172.21.15.102:0/2178225619 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe28061b80 con 0x7fbe140a2d30 2024-04-24T23:11:05.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbe140a2d30 msgr2=0x7fbe140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbe140a2d30 0x7fbe140a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbe281027e0 tx=0x7fbe1800b390 comp rx=0 tx=0).stop 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 msgr2=0x7fbe2819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 0x7fbe2819ef80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009dc0 tx=0x7fbe20003ba0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 shutdown_connections 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbe140a2d30 0x7fbe140a51f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe28103340 0x7fbe2819ef80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe28102980 0x7fbe28197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 --2- 172.21.15.102:0/2178225619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe28101790 0x7fbe281979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 >> 172.21.15.102:0/2178225619 conn(0x7fbe280fcd20 msgr2=0x7fbe28105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 shutdown_connections 2024-04-24T23:11:05.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.213+0000 7fbe2e15c640 1 -- 172.21.15.102:0/2178225619 wait complete. 2024-04-24T23:11:05.225 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.327+0000 7f1f2642a640 1 -- 172.21.15.102:0/4058783174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f20101780 msgr2=0x7f1f20101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.327+0000 7f1f2642a640 1 --2- 172.21.15.102:0/4058783174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f20101780 0x7f1f20101ba0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009900 tx=0x7f1f1402a410 comp rx=0 tx=0).stop 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.327+0000 7f1f2642a640 1 -- 172.21.15.102:0/4058783174 shutdown_connections 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.327+0000 7f1f2642a640 1 --2- 172.21.15.102:0/4058783174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20103330 0x7f1f20109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.327+0000 7f1f2642a640 1 --2- 172.21.15.102:0/4058783174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f20102970 0x7f1f20102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.327+0000 7f1f2642a640 1 --2- 172.21.15.102:0/4058783174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f20101780 0x7f1f20101ba0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.327+0000 7f1f2642a640 1 -- 172.21.15.102:0/4058783174 >> 172.21.15.102:0/4058783174 conn(0x7f1f200fccf0 msgr2=0x7f1f200ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.327+0000 7f1f2642a640 1 -- 172.21.15.102:0/4058783174 shutdown_connections 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.328+0000 7f1f2642a640 1 -- 172.21.15.102:0/4058783174 wait complete. 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.328+0000 7f1f2642a640 1 Processor -- start 2024-04-24T23:11:05.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.328+0000 7f1f2642a640 1 -- start start 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.328+0000 7f1f2642a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101780 0x7f1f20197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.328+0000 7f1f2642a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f20102970 0x7f1f20197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.328+0000 7f1f2642a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f20103330 0x7f1f2019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.328+0000 7f1f2642a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2010d300 con 0x7f1f20103330 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.328+0000 7f1f2642a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2010d180 con 0x7f1f20102970 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.328+0000 7f1f2642a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2010d480 con 0x7f1f20101780 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f1f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f20102970 0x7f1f20197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101780 0x7f1f20197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f1f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f20102970 0x7f1f20197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:60428/0 (socket says 172.21.15.102:60428) 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f249c7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f20103330 0x7f1f2019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f1f7fe640 1 -- 172.21.15.102:0/1236885677 learned_addr learned my addr 172.21.15.102:0/1236885677 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f249c7640 1 -- 172.21.15.102:0/1236885677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101780 msgr2=0x7f1f20197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f249c7640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101780 0x7f1f20197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f249c7640 1 -- 172.21.15.102:0/1236885677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f20102970 msgr2=0x7f1f20197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f249c7640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f20102970 0x7f1f20197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f249c7640 1 -- 172.21.15.102:0/1236885677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2019f680 con 0x7f1f20103330 2024-04-24T23:11:05.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.329+0000 7f1f249c7640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f20103330 0x7f1f2019f010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800be20 tx=0x7f1f1800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:05.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.331+0000 7f1f1d7fa640 1 -- 172.21.15.102:0/1236885677 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18019070 con 0x7f1f20103330 2024-04-24T23:11:05.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.331+0000 7f1f1d7fa640 1 -- 172.21.15.102:0/1236885677 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f180148f0 con 0x7f1f20103330 2024-04-24T23:11:05.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.331+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2019f970 con 0x7f1f20103330 2024-04-24T23:11:05.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.331+0000 7f1f1d7fa640 1 -- 172.21.15.102:0/1236885677 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180127f0 con 0x7f1f20103330 2024-04-24T23:11:05.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.331+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2019ff90 con 0x7f1f20103330 2024-04-24T23:11:05.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.332+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f20108bc0 con 0x7f1f20103330 2024-04-24T23:11:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.333+0000 7f1f1d7fa640 1 -- 172.21.15.102:0/1236885677 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1f1801c070 con 0x7f1f20103330 2024-04-24T23:11:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.334+0000 7f1f1d7fa640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1f080a2c10 0x7f1f080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.334+0000 7f1f1ffff640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1f080a2c10 0x7f1f080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.334+0000 7f1f1d7fa640 1 -- 172.21.15.102:0/1236885677 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1f180ce620 con 0x7f1f20103330 2024-04-24T23:11:05.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.334+0000 7f1f1ffff640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1f080a2c10 0x7f1f080a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1f14004770 tx=0x7f1f14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:05.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.338+0000 7f1f1d7fa640 1 -- 172.21.15.102:0/1236885677 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f1809b080 con 0x7f1f20103330 2024-04-24T23:11:05.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.540+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1f20061b80 con 0x7f1f080a2c10 2024-04-24T23:11:05.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.570+0000 7f1f1d7fa640 1 -- 172.21.15.102:0/1236885677 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1f20061b80 con 0x7f1f080a2c10 2024-04-24T23:11:05.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: pgmap v294: 97 pgs: 12 creating+peering, 71 active+clean, 14 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 622 B/s wr, 6 op/s 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3344544063' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] up:active 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: Cluster is now healthy 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2838790241' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2838790241' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] up:active 2024-04-24T23:11:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:11:05.573 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:11:05.190+0000 7f8a60dd8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:11:05.573 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:11:05.190+0000 7f8a60dd8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:11:05.573 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:11:05.190+0000 7f8a60dd8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:11:05.573 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:11:05.190+0000 7f8a60dd8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:11:05.573 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:11:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:11:05.190+0000 7f8a60dd8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:11:05.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.572+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1f080a2c10 msgr2=0x7f1f080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.572+0000 7f1f2642a640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1f080a2c10 0x7f1f080a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1f14004770 tx=0x7f1f14000bc0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.572+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f20103330 msgr2=0x7f1f2019f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.572+0000 7f1f2642a640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f20103330 0x7f1f2019f010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800be20 tx=0x7f1f1800bf10 comp rx=0 tx=0).stop 2024-04-24T23:11:05.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.572+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 shutdown_connections 2024-04-24T23:11:05.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.572+0000 7f1f2642a640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1f080a2c10 0x7f1f080a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.572+0000 7f1f2642a640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f20103330 0x7f1f2019f010 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.573+0000 7f1f2642a640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f20102970 0x7f1f20197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.573+0000 7f1f2642a640 1 --2- 172.21.15.102:0/1236885677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101780 0x7f1f20197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.573+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 >> 172.21.15.102:0/1236885677 conn(0x7f1f200fccf0 msgr2=0x7f1f20105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:05.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.573+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 shutdown_connections 2024-04-24T23:11:05.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.573+0000 7f1f2642a640 1 -- 172.21.15.102:0/1236885677 wait complete. 2024-04-24T23:11:05.585 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-24T23:11:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: pgmap v294: 97 pgs: 12 creating+peering, 71 active+clean, 14 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 622 B/s wr, 6 op/s 2024-04-24T23:11:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3344544063' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:11:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] up:active 2024-04-24T23:11:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:11:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:11:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: Cluster is now healthy 2024-04-24T23:11:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-24T23:11:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-24T23:11:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-24T23:11:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2838790241' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:11:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2838790241' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:11:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] up:active 2024-04-24T23:11:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:05 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:11:05.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.687+0000 7f2834e64640 1 -- 172.21.15.102:0/2020927782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2830102260 msgr2=0x7f2830104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.687+0000 7f2834e64640 1 --2- 172.21.15.102:0/2020927782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2830102260 0x7f2830104650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2820009900 tx=0x7f282002a410 comp rx=0 tx=0).stop 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.687+0000 7f2834e64640 1 -- 172.21.15.102:0/2020927782 shutdown_connections 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.687+0000 7f2834e64640 1 --2- 172.21.15.102:0/2020927782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28301074c0 0x7f28301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.687+0000 7f2834e64640 1 --2- 172.21.15.102:0/2020927782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830104b90 0x7f2830106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.687+0000 7f2834e64640 1 --2- 172.21.15.102:0/2020927782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2830102260 0x7f2830104650 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.687+0000 7f2834e64640 1 -- 172.21.15.102:0/2020927782 >> 172.21.15.102:0/2020927782 conn(0x7f28300fbbf0 msgr2=0x7f28300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.687+0000 7f2834e64640 1 -- 172.21.15.102:0/2020927782 shutdown_connections 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.687+0000 7f2834e64640 1 -- 172.21.15.102:0/2020927782 wait complete. 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f2834e64640 1 Processor -- start 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f2834e64640 1 -- start start 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f2834e64640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2830102260 0x7f2830101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f2834e64640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830104b90 0x7f2830100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f2834e64640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28301074c0 0x7f2830100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f2834e64640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f283010d320 con 0x7f28301074c0 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f2834e64640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f283010d1a0 con 0x7f2830104b90 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f2834e64640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f283010d4a0 con 0x7f2830102260 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f282ed9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28301074c0 0x7f2830100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f282ed9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28301074c0 0x7f2830100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40482/0 (socket says 172.21.15.102:40482) 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f282ed9d640 1 -- 172.21.15.102:0/1206384626 learned_addr learned my addr 172.21.15.102:0/1206384626 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f282dd9b640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830104b90 0x7f2830100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.688+0000 7f282e59c640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2830102260 0x7f2830101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f282ed9d640 1 -- 172.21.15.102:0/1206384626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2830102260 msgr2=0x7f2830101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f282ed9d640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2830102260 0x7f2830101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f282ed9d640 1 -- 172.21.15.102:0/1206384626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830104b90 msgr2=0x7f2830100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f282ed9d640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830104b90 0x7f2830100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f282ed9d640 1 -- 172.21.15.102:0/1206384626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2830100fe0 con 0x7f28301074c0 2024-04-24T23:11:05.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f282ed9d640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28301074c0 0x7f2830100750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f282800b970 tx=0x7f282800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:05.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f281f7fe640 1 -- 172.21.15.102:0/1206384626 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828019070 con 0x7f28301074c0 2024-04-24T23:11:05.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f281f7fe640 1 -- 172.21.15.102:0/1206384626 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2828014440 con 0x7f28301074c0 2024-04-24T23:11:05.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28301a1bb0 con 0x7f28301074c0 2024-04-24T23:11:05.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f281f7fe640 1 -- 172.21.15.102:0/1206384626 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28280075d0 con 0x7f28301074c0 2024-04-24T23:11:05.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.689+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28301a2150 con 0x7f28301074c0 2024-04-24T23:11:05.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.690+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28300fd6e0 con 0x7f28301074c0 2024-04-24T23:11:05.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.691+0000 7f281f7fe640 1 -- 172.21.15.102:0/1206384626 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f282801c070 con 0x7f28301074c0 2024-04-24T23:11:05.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.691+0000 7f281f7fe640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28180a2c10 0x7f28180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:05.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.691+0000 7f282e59c640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28180a2c10 0x7f28180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:05.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.691+0000 7f281f7fe640 1 -- 172.21.15.102:0/1206384626 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f28280ce440 con 0x7f28301074c0 2024-04-24T23:11:05.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.692+0000 7f282e59c640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28180a2c10 0x7f28180a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f282002a930 tx=0x7f2820000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:05.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.696+0000 7f281f7fe640 1 -- 172.21.15.102:0/1206384626 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f282809aea0 con 0x7f28301074c0 2024-04-24T23:11:05.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: pgmap v294: 97 pgs: 12 creating+peering, 71 active+clean, 14 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 622 B/s wr, 6 op/s 2024-04-24T23:11:05.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3344544063' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] up:active 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: Cluster is now healthy 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2838790241' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2838790241' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] up:active 2024-04-24T23:11:05.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:05 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:11:05.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.900+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2830061c10 con 0x7f28180a2c10 2024-04-24T23:11:05.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.906+0000 7f281f7fe640 1 -- 172.21.15.102:0/1206384626 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2830061c10 con 0x7f28180a2c10 2024-04-24T23:11:05.907 INFO:teuthology.orchestra.run.smithi102.stdout:/volumes/qa/sv_0/77976509-2cd8-41de-8489-e8b8dd738f18 2024-04-24T23:11:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28180a2c10 msgr2=0x7f28180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28180a2c10 0x7f28180a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f282002a930 tx=0x7f2820000bc0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28301074c0 msgr2=0x7f2830100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28301074c0 0x7f2830100750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f282800b970 tx=0x7f282800be30 comp rx=0 tx=0).stop 2024-04-24T23:11:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 shutdown_connections 2024-04-24T23:11:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28180a2c10 0x7f28180a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28301074c0 0x7f2830100750 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830104b90 0x7f2830100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 --2- 172.21.15.102:0/1206384626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2830102260 0x7f2830101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 >> 172.21.15.102:0/1206384626 conn(0x7f28300fbbf0 msgr2=0x7f2830102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 shutdown_connections 2024-04-24T23:11:05.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:05.908+0000 7f2834e64640 1 -- 172.21.15.102:0/1206384626 wait complete. 2024-04-24T23:11:05.920 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-24T23:11:06.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.021+0000 7f192b66b640 1 -- 172.21.15.102:0/1566354051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924102250 msgr2=0x7f1924108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.021+0000 7f192b66b640 1 --2- 172.21.15.102:0/1566354051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924102250 0x7f1924108ae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f191c009ca0 tx=0x7f191c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.021+0000 7f192b66b640 1 -- 172.21.15.102:0/1566354051 shutdown_connections 2024-04-24T23:11:06.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.021+0000 7f192b66b640 1 --2- 172.21.15.102:0/1566354051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924102250 0x7f1924108ae0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.021+0000 7f192b66b640 1 --2- 172.21.15.102:0/1566354051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1924101890 0x7f1924101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.021+0000 7f192b66b640 1 --2- 172.21.15.102:0/1566354051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19241006a0 0x7f1924100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.021+0000 7f192b66b640 1 -- 172.21.15.102:0/1566354051 >> 172.21.15.102:0/1566354051 conn(0x7f19240fbc10 msgr2=0x7f19240fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.021+0000 7f192b66b640 1 -- 172.21.15.102:0/1566354051 shutdown_connections 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.021+0000 7f192b66b640 1 -- 172.21.15.102:0/1566354051 wait complete. 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f192b66b640 1 Processor -- start 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f192b66b640 1 -- start start 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f192b66b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19241006a0 0x7f19241979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f192b66b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924101890 0x7f1924197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f192b66b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1924102250 0x7f192419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f192b66b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f192410d370 con 0x7f1924101890 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f192b66b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f192410d1f0 con 0x7f19241006a0 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f192b66b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f192410d4f0 con 0x7f1924102250 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f1928c06640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924101890 0x7f1924197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f1928c06640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924101890 0x7f1924197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40512/0 (socket says 172.21.15.102:40512) 2024-04-24T23:11:06.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f1928c06640 1 -- 172.21.15.102:0/141072430 learned_addr learned my addr 172.21.15.102:0/141072430 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f1929c08640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1924102250 0x7f192419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f1928c06640 1 -- 172.21.15.102:0/141072430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1924102250 msgr2=0x7f192419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f1928c06640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1924102250 0x7f192419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f1928c06640 1 -- 172.21.15.102:0/141072430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19241006a0 msgr2=0x7f19241979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f1928c06640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19241006a0 0x7f19241979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.022+0000 7f1928c06640 1 -- 172.21.15.102:0/141072430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192419f610 con 0x7f1924101890 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.023+0000 7f1928c06640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924101890 0x7f1924197f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f191400d770 tx=0x7f191400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.023+0000 7f191a7fc640 1 -- 172.21.15.102:0/141072430 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914017070 con 0x7f1924101890 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.023+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192419f900 con 0x7f1924101890 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.023+0000 7f191a7fc640 1 -- 172.21.15.102:0/141072430 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19140054d0 con 0x7f1924101890 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.023+0000 7f191a7fc640 1 -- 172.21.15.102:0/141072430 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914004cf0 con 0x7f1924101890 2024-04-24T23:11:06.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.023+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192419fe00 con 0x7f1924101890 2024-04-24T23:11:06.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.024+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1924107a40 con 0x7f1924101890 2024-04-24T23:11:06.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.025+0000 7f191a7fc640 1 -- 172.21.15.102:0/141072430 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1914004e90 con 0x7f1924101890 2024-04-24T23:11:06.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.025+0000 7f191a7fc640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19100a2b90 0x7f19100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.026+0000 7f1929407640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19100a2b90 0x7f19100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.026+0000 7f191a7fc640 1 -- 172.21.15.102:0/141072430 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f19140cea10 con 0x7f1924101890 2024-04-24T23:11:06.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.026+0000 7f1929407640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19100a2b90 0x7f19100a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f19241016f0 tx=0x7f1920006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:06.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.030+0000 7f191a7fc640 1 -- 172.21.15.102:0/141072430 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f191409bac0 con 0x7f1924101890 2024-04-24T23:11:06.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.230+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1924061b80 con 0x7f19100a2b90 2024-04-24T23:11:06.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.259+0000 7f191a7fc640 1 -- 172.21.15.102:0/141072430 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1924061b80 con 0x7f19100a2b90 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19100a2b90 msgr2=0x7f19100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19100a2b90 0x7f19100a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f19241016f0 tx=0x7f1920006c90 comp rx=0 tx=0).stop 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924101890 msgr2=0x7f1924197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924101890 0x7f1924197f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f191400d770 tx=0x7f191400dc30 comp rx=0 tx=0).stop 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 shutdown_connections 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19100a2b90 0x7f19100a5050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1924102250 0x7f192419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1924101890 0x7f1924197f30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 --2- 172.21.15.102:0/141072430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19241006a0 0x7f19241979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 >> 172.21.15.102:0/141072430 conn(0x7f19240fbc10 msgr2=0x7f1924104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:06.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 shutdown_connections 2024-04-24T23:11:06.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.261+0000 7f192b66b640 1 -- 172.21.15.102:0/141072430 wait complete. 2024-04-24T23:11:06.273 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-24T23:11:06.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.379+0000 7f1131709640 1 -- 172.21.15.102:0/1886209856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c109640 msgr2=0x7f112c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.379+0000 7f1131709640 1 --2- 172.21.15.102:0/1886209856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c109640 0x7f112c10bad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1124009900 tx=0x7f112402a480 comp rx=0 tx=0).stop 2024-04-24T23:11:06.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.379+0000 7f1131709640 1 -- 172.21.15.102:0/1886209856 shutdown_connections 2024-04-24T23:11:06.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.379+0000 7f1131709640 1 --2- 172.21.15.102:0/1886209856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c109640 0x7f112c10bad0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.379+0000 7f1131709640 1 --2- 172.21.15.102:0/1886209856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c074120 0x7f112c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.379+0000 7f1131709640 1 --2- 172.21.15.102:0/1886209856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c075700 0x7f112c073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.379+0000 7f1131709640 1 -- 172.21.15.102:0/1886209856 >> 172.21.15.102:0/1886209856 conn(0x7f112c0fcd20 msgr2=0x7f112c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.379+0000 7f1131709640 1 -- 172.21.15.102:0/1886209856 shutdown_connections 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.379+0000 7f1131709640 1 -- 172.21.15.102:0/1886209856 wait complete. 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f1131709640 1 Processor -- start 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f1131709640 1 -- start start 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f1131709640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c074120 0x7f112c199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f1131709640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c075700 0x7f112c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f1131709640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c109640 0x7f112c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f1131709640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f112c10f520 con 0x7f112c075700 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f1131709640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f112c10f3a0 con 0x7f112c074120 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f1131709640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f112c10f6a0 con 0x7f112c109640 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c075700 0x7f112c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c075700 0x7f112c19a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40538/0 (socket says 172.21.15.102:40538) 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112a7fc640 1 -- 172.21.15.102:0/985945356 learned_addr learned my addr 172.21.15.102:0/985945356 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:06.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112a7fc640 1 -- 172.21.15.102:0/985945356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c109640 msgr2=0x7f112c1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112b7fe640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c109640 0x7f112c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112affd640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c074120 0x7f112c199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112a7fc640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c109640 0x7f112c1a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112a7fc640 1 -- 172.21.15.102:0/985945356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c074120 msgr2=0x7f112c199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112a7fc640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c074120 0x7f112c199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.380+0000 7f112a7fc640 1 -- 172.21.15.102:0/985945356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c1a1830 con 0x7f112c075700 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.381+0000 7f112a7fc640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c075700 0x7f112c19a160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f112000b4a0 tx=0x7f112000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.381+0000 7f1113fff640 1 -- 172.21.15.102:0/985945356 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120014070 con 0x7f112c075700 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.381+0000 7f1113fff640 1 -- 172.21.15.102:0/985945356 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1120010ac0 con 0x7f112c075700 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.381+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112c1a1b20 con 0x7f112c075700 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.381+0000 7f1113fff640 1 -- 172.21.15.102:0/985945356 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120017c30 con 0x7f112c075700 2024-04-24T23:11:06.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.381+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f112c1a20c0 con 0x7f112c075700 2024-04-24T23:11:06.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.382+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f112c108400 con 0x7f112c075700 2024-04-24T23:11:06.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.383+0000 7f1113fff640 1 -- 172.21.15.102:0/985945356 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1120017070 con 0x7f112c075700 2024-04-24T23:11:06.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.384+0000 7f1113fff640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f11140a2ca0 0x7f11140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.384+0000 7f112affd640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f11140a2ca0 0x7f11140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.384+0000 7f1113fff640 1 -- 172.21.15.102:0/985945356 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f11200ce690 con 0x7f112c075700 2024-04-24T23:11:06.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.384+0000 7f112affd640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f11140a2ca0 0x7f11140a5160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f111c005c40 tx=0x7f111c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:06.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.388+0000 7f1113fff640 1 -- 172.21.15.102:0/985945356 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f112009b750 con 0x7f112c075700 2024-04-24T23:11:06.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:06 smithi102 ceph-mon[29216]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:06.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.591+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f112c061b80 con 0x7f11140a2ca0 2024-04-24T23:11:06.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.596+0000 7f1113fff640 1 -- 172.21.15.102:0/985945356 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f112c061b80 con 0x7f11140a2ca0 2024-04-24T23:11:06.598 INFO:teuthology.orchestra.run.smithi102.stdout:/volumes/qa/sv_1/328782d5-d730-4fef-95c8-551c912b3c86 2024-04-24T23:11:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f11140a2ca0 msgr2=0x7f11140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f11140a2ca0 0x7f11140a5160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f111c005c40 tx=0x7f111c005bd0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c075700 msgr2=0x7f112c19a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c075700 0x7f112c19a160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f112000b4a0 tx=0x7f112000b960 comp rx=0 tx=0).stop 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 shutdown_connections 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f11140a2ca0 0x7f11140a5160 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c109640 0x7f112c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c075700 0x7f112c19a160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 --2- 172.21.15.102:0/985945356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c074120 0x7f112c199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 >> 172.21.15.102:0/985945356 conn(0x7f112c0fcd20 msgr2=0x7f112c1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 shutdown_connections 2024-04-24T23:11:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.598+0000 7f1131709640 1 -- 172.21.15.102:0/985945356 wait complete. 2024-04-24T23:11:06.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:06 smithi161 ceph-mon[32709]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:06.609 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:11:06 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b[63009]: 2024-04-24T23:11:06.289+0000 7ff14407f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:11:06.609 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:11:06 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b[63009]: 2024-04-24T23:11:06.289+0000 7ff14407f700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-24T23:11:06.610 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:11:06 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b[63009]: 2024-04-24T23:11:06.289+0000 7ff14407f700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-24T23:11:06.611 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.707+0000 7f9513cd4640 1 -- 172.21.15.102:0/3530183997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c101790 msgr2=0x7f950c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.707+0000 7f9513cd4640 1 --2- 172.21.15.102:0/3530183997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c101790 0x7f950c101bb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9508009900 tx=0x7f950802a410 comp rx=0 tx=0).stop 2024-04-24T23:11:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.707+0000 7f9513cd4640 1 -- 172.21.15.102:0/3530183997 shutdown_connections 2024-04-24T23:11:06.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.707+0000 7f9513cd4640 1 --2- 172.21.15.102:0/3530183997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f950c103340 0x7f950c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.707+0000 7f9513cd4640 1 --2- 172.21.15.102:0/3530183997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950c102980 0x7f950c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.707+0000 7f9513cd4640 1 --2- 172.21.15.102:0/3530183997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c101790 0x7f950c101bb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.707+0000 7f9513cd4640 1 -- 172.21.15.102:0/3530183997 >> 172.21.15.102:0/3530183997 conn(0x7f950c0fcd20 msgr2=0x7f950c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.707+0000 7f9513cd4640 1 -- 172.21.15.102:0/3530183997 shutdown_connections 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.707+0000 7f9513cd4640 1 -- 172.21.15.102:0/3530183997 wait complete. 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9513cd4640 1 Processor -- start 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9513cd4640 1 -- start start 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9513cd4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f950c101790 0x7f950c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9513cd4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c102980 0x7f950c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9513cd4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950c103340 0x7f950c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9513cd4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f950c10d390 con 0x7f950c102980 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9513cd4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f950c10d210 con 0x7f950c101790 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9513cd4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f950c10d510 con 0x7f950c103340 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f951126f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c102980 0x7f950c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f951126f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c102980 0x7f950c073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55652/0 (socket says 172.21.15.102:55652) 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f951126f640 1 -- 172.21.15.102:0/368718556 learned_addr learned my addr 172.21.15.102:0/368718556 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9511a70640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f950c101790 0x7f950c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f951126f640 1 -- 172.21.15.102:0/368718556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950c103340 msgr2=0x7f950c074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f951126f640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950c103340 0x7f950c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f951126f640 1 -- 172.21.15.102:0/368718556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f950c101790 msgr2=0x7f950c0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f951126f640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f950c101790 0x7f950c0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f951126f640 1 -- 172.21.15.102:0/368718556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950c074900 con 0x7f950c102980 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9511a70640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f950c101790 0x7f950c0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f951126f640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c102980 0x7f950c073b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00b4a0 tx=0x7f94fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9502ffd640 1 -- 172.21.15.102:0/368718556 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc014070 con 0x7f950c102980 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950c1a1c70 con 0x7f950c102980 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9502ffd640 1 -- 172.21.15.102:0/368718556 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94fc010ac0 con 0x7f950c102980 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.708+0000 7f9502ffd640 1 -- 172.21.15.102:0/368718556 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc017c30 con 0x7f950c102980 2024-04-24T23:11:06.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.709+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950c1a2240 con 0x7f950c102980 2024-04-24T23:11:06.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.710+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950c108b70 con 0x7f950c102980 2024-04-24T23:11:06.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.710+0000 7f9502ffd640 1 -- 172.21.15.102:0/368718556 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f94fc017070 con 0x7f950c102980 2024-04-24T23:11:06.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.711+0000 7f9502ffd640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94f80a2c20 0x7f94f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.711+0000 7f9511a70640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94f80a2c20 0x7f94f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.711+0000 7f9502ffd640 1 -- 172.21.15.102:0/368718556 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f94fc0ce6c0 con 0x7f950c102980 2024-04-24T23:11:06.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.712+0000 7f9511a70640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94f80a2c20 0x7f94f80a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f9508031d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:06.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.715+0000 7f9502ffd640 1 -- 172.21.15.102:0/368718556 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f94fc09b780 con 0x7f950c102980 2024-04-24T23:11:06.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:06 smithi183 ceph-mon[33137]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:06.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.926+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f950c02d320 con 0x7f950c102980 2024-04-24T23:11:06.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.927+0000 7f9502ffd640 1 -- 172.21.15.102:0/368718556 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7f94fc09b080 con 0x7f950c102980 2024-04-24T23:11:06.929 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:11:06.929 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:11:00.643113+0000","modified":"2024-04-24T23:11:04.856565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/1258581930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1258581930},{"type":"v1","addr":"172.21.15.161:6839","nonce":1258581930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24473,24497,24472]},"id":1}]} 2024-04-24T23:11:06.929 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.928+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94f80a2c20 msgr2=0x7f94f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.928+0000 7f9513cd4640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94f80a2c20 0x7f94f80a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f9508031d10 comp rx=0 tx=0).stop 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.928+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c102980 msgr2=0x7f950c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.929+0000 7f9513cd4640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c102980 0x7f950c073b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00b4a0 tx=0x7f94fc00b960 comp rx=0 tx=0).stop 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.929+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 shutdown_connections 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.929+0000 7f9513cd4640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94f80a2c20 0x7f94f80a50e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.929+0000 7f9513cd4640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950c103340 0x7f950c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.929+0000 7f9513cd4640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f950c102980 0x7f950c073b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.929+0000 7f9513cd4640 1 --2- 172.21.15.102:0/368718556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f950c101790 0x7f950c0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.929+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 >> 172.21.15.102:0/368718556 conn(0x7f950c0fcd20 msgr2=0x7f950c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.929+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 shutdown_connections 2024-04-24T23:11:06.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:06.929+0000 7f9513cd4640 1 -- 172.21.15.102:0/368718556 wait complete. 2024-04-24T23:11:06.942 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:07.040 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.036+0000 7ffbbfded640 1 -- 172.21.15.102:0/2852292259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8100680 msgr2=0x7ffbb8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.036+0000 7ffbbfded640 1 --2- 172.21.15.102:0/2852292259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8100680 0x7ffbb8100aa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009900 tx=0x7ffba802a410 comp rx=0 tx=0).stop 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.036+0000 7ffbbfded640 1 -- 172.21.15.102:0/2852292259 shutdown_connections 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.036+0000 7ffbbfded640 1 --2- 172.21.15.102:0/2852292259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8102230 0x7ffbb8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.036+0000 7ffbbfded640 1 --2- 172.21.15.102:0/2852292259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb8101870 0x7ffbb8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.036+0000 7ffbbfded640 1 --2- 172.21.15.102:0/2852292259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8100680 0x7ffbb8100aa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 -- 172.21.15.102:0/2852292259 >> 172.21.15.102:0/2852292259 conn(0x7ffbb80fbbf0 msgr2=0x7ffbb80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 -- 172.21.15.102:0/2852292259 shutdown_connections 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 -- 172.21.15.102:0/2852292259 wait complete. 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 Processor -- start 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 -- start start 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb8100680 0x7ffbb8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8101870 0x7ffbb8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102230 0x7ffbb819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb810d2d0 con 0x7ffbb8102230 2024-04-24T23:11:07.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb810d150 con 0x7ffbb8100680 2024-04-24T23:11:07.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbfded640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb810d450 con 0x7ffbb8101870 2024-04-24T23:11:07.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbe38a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102230 0x7ffbb819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbe38a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102230 0x7ffbb819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55670/0 (socket says 172.21.15.102:55670) 2024-04-24T23:11:07.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.037+0000 7ffbbe38a640 1 -- 172.21.15.102:0/3961857200 learned_addr learned my addr 172.21.15.102:0/3961857200 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:07.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbd388640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8101870 0x7ffbb8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbdb89640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb8100680 0x7ffbb8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbe38a640 1 -- 172.21.15.102:0/3961857200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8101870 msgr2=0x7ffbb8197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbe38a640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8101870 0x7ffbb8197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbe38a640 1 -- 172.21.15.102:0/3961857200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb8100680 msgr2=0x7ffbb8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbe38a640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb8100680 0x7ffbb8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbe38a640 1 -- 172.21.15.102:0/3961857200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb819f5b0 con 0x7ffbb8102230 2024-04-24T23:11:07.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbdb89640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb8100680 0x7ffbb8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:07.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbe38a640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102230 0x7ffbb819ef40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000cad0 tx=0x7ffbb000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbaeffd640 1 -- 172.21.15.102:0/3961857200 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000ad60 con 0x7ffbb8102230 2024-04-24T23:11:07.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbaeffd640 1 -- 172.21.15.102:0/3961857200 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbb0005020 con 0x7ffbb8102230 2024-04-24T23:11:07.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb819f8a0 con 0x7ffbb8102230 2024-04-24T23:11:07.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbaeffd640 1 -- 172.21.15.102:0/3961857200 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000f6b0 con 0x7ffbb8102230 2024-04-24T23:11:07.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.038+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb819fdf0 con 0x7ffbb8102230 2024-04-24T23:11:07.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.039+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb8107a70 con 0x7ffbb8102230 2024-04-24T23:11:07.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.040+0000 7ffbaeffd640 1 -- 172.21.15.102:0/3961857200 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ffbb00051d0 con 0x7ffbb8102230 2024-04-24T23:11:07.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.041+0000 7ffbaeffd640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffba40a2ca0 0x7ffba40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.041+0000 7ffbbdb89640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffba40a2ca0 0x7ffba40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.041+0000 7ffbaeffd640 1 -- 172.21.15.102:0/3961857200 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffbb00cf010 con 0x7ffbb8102230 2024-04-24T23:11:07.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.041+0000 7ffbbdb89640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffba40a2ca0 0x7ffba40a5160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffba8000c00 tx=0x7ffba8031d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.045+0000 7ffbaeffd640 1 -- 172.21.15.102:0/3961857200 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffbb0014030 con 0x7ffbb8102230 2024-04-24T23:11:07.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.222+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbb802d2f0 con 0x7ffbb8102230 2024-04-24T23:11:07.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.223+0000 7ffbaeffd640 1 -- 172.21.15.102:0/3961857200 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22279 (secure 0 0 0) 0x7ffbb009bfc0 con 0x7ffbb8102230 2024-04-24T23:11:07.226 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:11:07.226 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":86,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:11:02.189206+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:07.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffba40a2ca0 msgr2=0x7ffba40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffba40a2ca0 0x7ffba40a5160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffba8000c00 tx=0x7ffba8031d10 comp rx=0 tx=0).stop 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102230 msgr2=0x7ffbb819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102230 0x7ffbb819ef40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000cad0 tx=0x7ffbb000ced0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 shutdown_connections 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ffba40a2ca0 0x7ffba40a5160 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102230 0x7ffbb819ef40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8101870 0x7ffbb8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 --2- 172.21.15.102:0/3961857200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb8100680 0x7ffbb8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 >> 172.21.15.102:0/3961857200 conn(0x7ffbb80fbbf0 msgr2=0x7ffbb8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 shutdown_connections 2024-04-24T23:11:07.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.226+0000 7ffbbfded640 1 -- 172.21.15.102:0/3961857200 wait complete. 2024-04-24T23:11:07.239 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:11:07.239 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:11:07.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2672902512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc102220 msgr2=0x7f75cc104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2672902512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc102220 0x7f75cc104610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009900 tx=0x7f75b402a410 comp rx=0 tx=0).stop 2024-04-24T23:11:07.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2672902512 shutdown_connections 2024-04-24T23:11:07.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2672902512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc107480 0x7f75cc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2672902512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75cc104b50 0x7f75cc106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2672902512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc102220 0x7f75cc104610 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2672902512 >> 172.21.15.102:0/2672902512 conn(0x7f75cc0fbc30 msgr2=0x7f75cc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2672902512 shutdown_connections 2024-04-24T23:11:07.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2672902512 wait complete. 2024-04-24T23:11:07.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 Processor -- start 2024-04-24T23:11:07.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 -- start start 2024-04-24T23:11:07.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc102220 0x7f75cc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75cc104b50 0x7f75cc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc107480 0x7f75cc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d350 con 0x7f75cc107480 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d1d0 con 0x7f75cc102220 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d2ca3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d4d0 con 0x7f75cc104b50 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d1240640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc107480 0x7f75cc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.338+0000 7f75d1240640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc107480 0x7f75cc19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55684/0 (socket says 172.21.15.102:55684) 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d1240640 1 -- 172.21.15.102:0/2667264318 learned_addr learned my addr 172.21.15.102:0/2667264318 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75cbfff640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75cc104b50 0x7f75cc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d0a3f640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc102220 0x7f75cc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d1240640 1 -- 172.21.15.102:0/2667264318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75cc104b50 msgr2=0x7f75cc197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d1240640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75cc104b50 0x7f75cc197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d1240640 1 -- 172.21.15.102:0/2667264318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc102220 msgr2=0x7f75cc197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d1240640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc102220 0x7f75cc197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d1240640 1 -- 172.21.15.102:0/2667264318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc19f680 con 0x7f75cc107480 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75cbfff640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75cc104b50 0x7f75cc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d1240640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc107480 0x7f75cc19f010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f75c400eb10 tx=0x7f75c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75c9ffb640 1 -- 172.21.15.102:0/2667264318 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4004b20 con 0x7f75cc107480 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75c9ffb640 1 -- 172.21.15.102:0/2667264318 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75c4004cc0 con 0x7f75cc107480 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc19f970 con 0x7f75cc107480 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75c9ffb640 1 -- 172.21.15.102:0/2667264318 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4010690 con 0x7f75cc107480 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.339+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc19fec0 con 0x7f75cc107480 2024-04-24T23:11:07.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.340+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75cc0fd720 con 0x7f75cc107480 2024-04-24T23:11:07.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.341+0000 7f75c9ffb640 1 -- 172.21.15.102:0/2667264318 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f75c4005020 con 0x7f75cc107480 2024-04-24T23:11:07.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.342+0000 7f75c9ffb640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75b80a2ca0 0x7f75b80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.342+0000 7f75d0a3f640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75b80a2ca0 0x7f75b80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.342+0000 7f75c9ffb640 1 -- 172.21.15.102:0/2667264318 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f75c40cf6a0 con 0x7f75cc107480 2024-04-24T23:11:07.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.342+0000 7f75d0a3f640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75b80a2ca0 0x7f75b80a5160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f75b4000c00 tx=0x7f75b40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.346+0000 7f75c9ffb640 1 -- 172.21.15.102:0/2667264318 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f75c409c650 con 0x7f75cc107480 2024-04-24T23:11:07.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.519+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75cc061b80 con 0x7f75b80a2ca0 2024-04-24T23:11:07.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.527+0000 7f75c9ffb640 1 -- 172.21.15.102:0/2667264318 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7f75cc061b80 con 0x7f75b80a2ca0 2024-04-24T23:11:07.532 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:11:07.533 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-24T23:11:07.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75b80a2ca0 msgr2=0x7f75b80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75b80a2ca0 0x7f75b80a5160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f75b4000c00 tx=0x7f75b40314a0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc107480 msgr2=0x7f75cc19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc107480 0x7f75cc19f010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f75c400eb10 tx=0x7f75c400c2d0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 shutdown_connections 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75b80a2ca0 0x7f75b80a5160 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75cc107480 0x7f75cc19f010 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75cc104b50 0x7f75cc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 --2- 172.21.15.102:0/2667264318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75cc102220 0x7f75cc197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 >> 172.21.15.102:0/2667264318 conn(0x7f75cc0fbc30 msgr2=0x7f75cc102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 shutdown_connections 2024-04-24T23:11:07.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.530+0000 7f75d2ca3640 1 -- 172.21.15.102:0/2667264318 wait complete. 2024-04-24T23:11:07.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:07 smithi102 ceph-mon[29216]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:07.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:07 smithi102 ceph-mon[29216]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:07.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:07 smithi102 ceph-mon[29216]: pgmap v295: 97 pgs: 97 active+clean; 586 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-24T23:11:07.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:07 smithi102 ceph-mon[29216]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:07.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:07 smithi102 ceph-mon[29216]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:11:07.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:07 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/368718556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.542 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:07 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3961857200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.543 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-04-24T23:11:06.218153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600140,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":144,"num_read_kb":126,"num_write":262,"num_write_kb":2963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356072,"kb_used_data":11020,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585752,"statfs":{"total":1151940427776,"available":1151575810048,"internally_reserved":0,"allocated":11284480,"data_stored":7024661,"data_compressed":106857,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9772,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":81,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001364"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618869+0000","last_change":"2024-04-24T23:11:01.217761+0000","last_active":"2024-04-24T23:11:02.618869+0000","last_peered":"2024-04-24T23:11:02.618869+0000","last_clean":"2024-04-24T23:11:02.618869+0000","last_became_active":"2024-04-24T23:11:01.217330+0000","last_became_peered":"2024-04-24T23:11:01.217330+0000","last_unstale":"2024-04-24T23:11:02.618869+0000","last_undegraded":"2024-04-24T23:11:02.618869+0000","last_fullsized":"2024-04-24T23:11:02.618869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:10.690940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617991+0000","last_change":"2024-04-24T23:11:01.206722+0000","last_active":"2024-04-24T23:11:02.617991+0000","last_peered":"2024-04-24T23:11:02.617991+0000","last_clean":"2024-04-24T23:11:02.617991+0000","last_became_active":"2024-04-24T23:11:01.205573+0000","last_became_peered":"2024-04-24T23:11:01.205573+0000","last_unstale":"2024-04-24T23:11:02.617991+0000","last_undegraded":"2024-04-24T23:11:02.617991+0000","last_fullsized":"2024-04-24T23:11:02.617991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:01.757792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617957+0000","last_change":"2024-04-24T23:11:01.213159+0000","last_active":"2024-04-24T23:11:02.617957+0000","last_peered":"2024-04-24T23:11:02.617957+0000","last_clean":"2024-04-24T23:11:02.617957+0000","last_became_active":"2024-04-24T23:11:01.212811+0000","last_became_peered":"2024-04-24T23:11:01.212811+0000","last_unstale":"2024-04-24T23:11:02.617957+0000","last_undegraded":"2024-04-24T23:11:02.617957+0000","last_fullsized":"2024-04-24T23:11:02.617957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:02.053206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.215153+0000","last_change":"2024-04-24T23:11:01.213023+0000","last_active":"2024-04-24T23:11:01.215153+0000","last_peered":"2024-04-24T23:11:01.215153+0000","last_clean":"2024-04-24T23:11:01.215153+0000","last_became_active":"2024-04-24T23:11:01.212689+0000","last_became_peered":"2024-04-24T23:11:01.212689+0000","last_unstale":"2024-04-24T23:11:01.215153+0000","last_undegraded":"2024-04-24T23:11:01.215153+0000","last_fullsized":"2024-04-24T23:11:01.215153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:00.942199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618302+0000","last_change":"2024-04-24T23:11:01.210845+0000","last_active":"2024-04-24T23:11:02.618302+0000","last_peered":"2024-04-24T23:11:02.618302+0000","last_clean":"2024-04-24T23:11:02.618302+0000","last_became_active":"2024-04-24T23:11:01.210389+0000","last_became_peered":"2024-04-24T23:11:01.210389+0000","last_unstale":"2024-04-24T23:11:02.618302+0000","last_undegraded":"2024-04-24T23:11:02.618302+0000","last_fullsized":"2024-04-24T23:11:02.618302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:03.946294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618518+0000","last_change":"2024-04-24T23:11:01.203082+0000","last_active":"2024-04-24T23:11:02.618518+0000","last_peered":"2024-04-24T23:11:02.618518+0000","last_clean":"2024-04-24T23:11:02.618518+0000","last_became_active":"2024-04-24T23:11:01.202605+0000","last_became_peered":"2024-04-24T23:11:01.202605+0000","last_unstale":"2024-04-24T23:11:02.618518+0000","last_undegraded":"2024-04-24T23:11:02.618518+0000","last_fullsized":"2024-04-24T23:11:02.618518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:29.248361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618138+0000","last_change":"2024-04-24T23:11:01.213002+0000","last_active":"2024-04-24T23:11:02.618138+0000","last_peered":"2024-04-24T23:11:02.618138+0000","last_clean":"2024-04-24T23:11:02.618138+0000","last_became_active":"2024-04-24T23:11:01.212690+0000","last_became_peered":"2024-04-24T23:11:01.212690+0000","last_unstale":"2024-04-24T23:11:02.618138+0000","last_undegraded":"2024-04-24T23:11:02.618138+0000","last_fullsized":"2024-04-24T23:11:02.618138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:06.658975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196454+0000","last_change":"2024-04-24T23:11:01.213041+0000","last_active":"2024-04-24T23:11:02.196454+0000","last_peered":"2024-04-24T23:11:02.196454+0000","last_clean":"2024-04-24T23:11:02.196454+0000","last_became_active":"2024-04-24T23:11:01.212659+0000","last_became_peered":"2024-04-24T23:11:01.212659+0000","last_unstale":"2024-04-24T23:11:02.196454+0000","last_undegraded":"2024-04-24T23:11:02.196454+0000","last_fullsized":"2024-04-24T23:11:02.196454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:53.229204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.217998+0000","last_change":"2024-04-24T23:11:01.217638+0000","last_active":"2024-04-24T23:11:01.217998+0000","last_peered":"2024-04-24T23:11:01.217998+0000","last_clean":"2024-04-24T23:11:01.217998+0000","last_became_active":"2024-04-24T23:11:01.217288+0000","last_became_peered":"2024-04-24T23:11:01.217288+0000","last_unstale":"2024-04-24T23:11:01.217998+0000","last_undegraded":"2024-04-24T23:11:01.217998+0000","last_fullsized":"2024-04-24T23:11:01.217998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:04.113776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618647+0000","last_change":"2024-04-24T23:11:01.203188+0000","last_active":"2024-04-24T23:11:02.618647+0000","last_peered":"2024-04-24T23:11:02.618647+0000","last_clean":"2024-04-24T23:11:02.618647+0000","last_became_active":"2024-04-24T23:11:01.202763+0000","last_became_peered":"2024-04-24T23:11:01.202763+0000","last_unstale":"2024-04-24T23:11:02.618647+0000","last_undegraded":"2024-04-24T23:11:02.618647+0000","last_fullsized":"2024-04-24T23:11:02.618647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:04.414650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618545+0000","last_change":"2024-04-24T23:11:01.220641+0000","last_active":"2024-04-24T23:11:02.618545+0000","last_peered":"2024-04-24T23:11:02.618545+0000","last_clean":"2024-04-24T23:11:02.618545+0000","last_became_active":"2024-04-24T23:11:01.220344+0000","last_became_peered":"2024-04-24T23:11:01.220344+0000","last_unstale":"2024-04-24T23:11:02.618545+0000","last_undegraded":"2024-04-24T23:11:02.618545+0000","last_fullsized":"2024-04-24T23:11:02.618545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:44.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.210929+0000","last_change":"2024-04-24T23:11:01.209805+0000","last_active":"2024-04-24T23:11:01.210929+0000","last_peered":"2024-04-24T23:11:01.210929+0000","last_clean":"2024-04-24T23:11:01.210929+0000","last_became_active":"2024-04-24T23:11:01.209497+0000","last_became_peered":"2024-04-24T23:11:01.209497+0000","last_unstale":"2024-04-24T23:11:01.210929+0000","last_undegraded":"2024-04-24T23:11:01.210929+0000","last_fullsized":"2024-04-24T23:11:01.210929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.930210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.210893+0000","last_change":"2024-04-24T23:11:01.205763+0000","last_active":"2024-04-24T23:11:01.210893+0000","last_peered":"2024-04-24T23:11:01.210893+0000","last_clean":"2024-04-24T23:11:01.210893+0000","last_became_active":"2024-04-24T23:11:01.205402+0000","last_became_peered":"2024-04-24T23:11:01.205402+0000","last_unstale":"2024-04-24T23:11:01.210893+0000","last_undegraded":"2024-04-24T23:11:01.210893+0000","last_fullsized":"2024-04-24T23:11:01.210893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:27.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617492+0000","last_change":"2024-04-24T23:11:01.209466+0000","last_active":"2024-04-24T23:11:02.617492+0000","last_peered":"2024-04-24T23:11:02.617492+0000","last_clean":"2024-04-24T23:11:02.617492+0000","last_became_active":"2024-04-24T23:11:01.208946+0000","last_became_peered":"2024-04-24T23:11:01.208946+0000","last_unstale":"2024-04-24T23:11:02.617492+0000","last_undegraded":"2024-04-24T23:11:02.617492+0000","last_fullsized":"2024-04-24T23:11:02.617492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:07.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617512+0000","last_change":"2024-04-24T23:11:01.215170+0000","last_active":"2024-04-24T23:11:02.617512+0000","last_peered":"2024-04-24T23:11:02.617512+0000","last_clean":"2024-04-24T23:11:02.617512+0000","last_became_active":"2024-04-24T23:11:01.214867+0000","last_became_peered":"2024-04-24T23:11:01.214867+0000","last_unstale":"2024-04-24T23:11:02.617512+0000","last_undegraded":"2024-04-24T23:11:02.617512+0000","last_fullsized":"2024-04-24T23:11:02.617512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.830234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.217138+0000","last_change":"2024-04-24T23:11:01.216691+0000","last_active":"2024-04-24T23:11:01.217138+0000","last_peered":"2024-04-24T23:11:01.217138+0000","last_clean":"2024-04-24T23:11:01.217138+0000","last_became_active":"2024-04-24T23:11:01.216407+0000","last_became_peered":"2024-04-24T23:11:01.216407+0000","last_unstale":"2024-04-24T23:11:01.217138+0000","last_undegraded":"2024-04-24T23:11:01.217138+0000","last_fullsized":"2024-04-24T23:11:01.217138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:13.289291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618946+0000","last_change":"2024-04-24T23:11:01.217764+0000","last_active":"2024-04-24T23:11:02.618946+0000","last_peered":"2024-04-24T23:11:02.618946+0000","last_clean":"2024-04-24T23:11:02.618946+0000","last_became_active":"2024-04-24T23:11:01.217424+0000","last_became_peered":"2024-04-24T23:11:01.217424+0000","last_unstale":"2024-04-24T23:11:02.618946+0000","last_undegraded":"2024-04-24T23:11:02.618946+0000","last_fullsized":"2024-04-24T23:11:02.618946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:11.517317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618394+0000","last_change":"2024-04-24T23:11:01.214351+0000","last_active":"2024-04-24T23:11:02.618394+0000","last_peered":"2024-04-24T23:11:02.618394+0000","last_clean":"2024-04-24T23:11:02.618394+0000","last_became_active":"2024-04-24T23:11:01.214034+0000","last_became_peered":"2024-04-24T23:11:01.214034+0000","last_unstale":"2024-04-24T23:11:02.618394+0000","last_undegraded":"2024-04-24T23:11:02.618394+0000","last_fullsized":"2024-04-24T23:11:02.618394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:26.105888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.316062+0000","last_change":"2024-04-24T23:11:01.212836+0000","last_active":"2024-04-24T23:11:02.316062+0000","last_peered":"2024-04-24T23:11:02.316062+0000","last_clean":"2024-04-24T23:11:02.316062+0000","last_became_active":"2024-04-24T23:11:01.209223+0000","last_became_peered":"2024-04-24T23:11:01.209223+0000","last_unstale":"2024-04-24T23:11:02.316062+0000","last_undegraded":"2024-04-24T23:11:02.316062+0000","last_fullsized":"2024-04-24T23:11:02.316062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:40.100399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.343540+0000","last_change":"2024-04-24T23:11:00.302055+0000","last_active":"2024-04-24T23:11:04.343540+0000","last_peered":"2024-04-24T23:11:04.343540+0000","last_clean":"2024-04-24T23:11:04.343540+0000","last_became_active":"2024-04-24T23:11:00.301726+0000","last_became_peered":"2024-04-24T23:11:00.301726+0000","last_unstale":"2024-04-24T23:11:04.343540+0000","last_undegraded":"2024-04-24T23:11:04.343540+0000","last_fullsized":"2024-04-24T23:11:04.343540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:54.039099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196683+0000","last_change":"2024-04-24T23:11:01.219527+0000","last_active":"2024-04-24T23:11:02.196683+0000","last_peered":"2024-04-24T23:11:02.196683+0000","last_clean":"2024-04-24T23:11:02.196683+0000","last_became_active":"2024-04-24T23:11:01.219196+0000","last_became_peered":"2024-04-24T23:11:01.219196+0000","last_unstale":"2024-04-24T23:11:02.196683+0000","last_undegraded":"2024-04-24T23:11:02.196683+0000","last_fullsized":"2024-04-24T23:11:02.196683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:58.584402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196796+0000","last_change":"2024-04-24T23:11:00.309260+0000","last_active":"2024-04-24T23:11:02.196796+0000","last_peered":"2024-04-24T23:11:02.196796+0000","last_clean":"2024-04-24T23:11:02.196796+0000","last_became_active":"2024-04-24T23:11:00.308969+0000","last_became_peered":"2024-04-24T23:11:00.308969+0000","last_unstale":"2024-04-24T23:11:02.196796+0000","last_undegraded":"2024-04-24T23:11:02.196796+0000","last_fullsized":"2024-04-24T23:11:02.196796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:23.137469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618171+0000","last_change":"2024-04-24T23:11:01.216515+0000","last_active":"2024-04-24T23:11:02.618171+0000","last_peered":"2024-04-24T23:11:02.618171+0000","last_clean":"2024-04-24T23:11:02.618171+0000","last_became_active":"2024-04-24T23:11:01.216168+0000","last_became_peered":"2024-04-24T23:11:01.216168+0000","last_unstale":"2024-04-24T23:11:02.618171+0000","last_undegraded":"2024-04-24T23:11:02.618171+0000","last_fullsized":"2024-04-24T23:11:02.618171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:05.229365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618850+0000","last_change":"2024-04-24T23:11:00.302558+0000","last_active":"2024-04-24T23:11:02.618850+0000","last_peered":"2024-04-24T23:11:02.618850+0000","last_clean":"2024-04-24T23:11:02.618850+0000","last_became_active":"2024-04-24T23:11:00.302162+0000","last_became_peered":"2024-04-24T23:11:00.302162+0000","last_unstale":"2024-04-24T23:11:02.618850+0000","last_undegraded":"2024-04-24T23:11:02.618850+0000","last_fullsized":"2024-04-24T23:11:02.618850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:52.623679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.222961+0000","last_change":"2024-04-24T23:11:01.222129+0000","last_active":"2024-04-24T23:11:01.222961+0000","last_peered":"2024-04-24T23:11:01.222961+0000","last_clean":"2024-04-24T23:11:01.222961+0000","last_became_active":"2024-04-24T23:11:01.221795+0000","last_became_peered":"2024-04-24T23:11:01.221795+0000","last_unstale":"2024-04-24T23:11:01.222961+0000","last_undegraded":"2024-04-24T23:11:01.222961+0000","last_fullsized":"2024-04-24T23:11:01.222961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:35.194854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618553+0000","last_change":"2024-04-24T23:11:00.308586+0000","last_active":"2024-04-24T23:11:02.618553+0000","last_peered":"2024-04-24T23:11:02.618553+0000","last_clean":"2024-04-24T23:11:02.618553+0000","last_became_active":"2024-04-24T23:11:00.305054+0000","last_became_peered":"2024-04-24T23:11:00.305054+0000","last_unstale":"2024-04-24T23:11:02.618553+0000","last_undegraded":"2024-04-24T23:11:02.618553+0000","last_fullsized":"2024-04-24T23:11:02.618553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:11.371829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.212468+0000","last_change":"2024-04-24T23:11:01.211868+0000","last_active":"2024-04-24T23:11:01.212468+0000","last_peered":"2024-04-24T23:11:01.212468+0000","last_clean":"2024-04-24T23:11:01.212468+0000","last_became_active":"2024-04-24T23:11:01.211504+0000","last_became_peered":"2024-04-24T23:11:01.211504+0000","last_unstale":"2024-04-24T23:11:01.212468+0000","last_undegraded":"2024-04-24T23:11:01.212468+0000","last_fullsized":"2024-04-24T23:11:01.212468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:47.974137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.302721+0000","last_change":"2024-04-24T23:11:00.302389+0000","last_active":"2024-04-24T23:11:01.302721+0000","last_peered":"2024-04-24T23:11:01.302721+0000","last_clean":"2024-04-24T23:11:01.302721+0000","last_became_active":"2024-04-24T23:11:00.302061+0000","last_became_peered":"2024-04-24T23:11:00.302061+0000","last_unstale":"2024-04-24T23:11:01.302721+0000","last_undegraded":"2024-04-24T23:11:01.302721+0000","last_fullsized":"2024-04-24T23:11:01.302721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:28.437391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618464+0000","last_change":"2024-04-24T23:11:01.202389+0000","last_active":"2024-04-24T23:11:02.618464+0000","last_peered":"2024-04-24T23:11:02.618464+0000","last_clean":"2024-04-24T23:11:02.618464+0000","last_became_active":"2024-04-24T23:11:01.201986+0000","last_became_peered":"2024-04-24T23:11:01.201986+0000","last_unstale":"2024-04-24T23:11:02.618464+0000","last_undegraded":"2024-04-24T23:11:02.618464+0000","last_fullsized":"2024-04-24T23:11:02.618464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:25.743284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.343040+0000","last_change":"2024-04-24T23:11:00.307278+0000","last_active":"2024-04-24T23:11:04.343040+0000","last_peered":"2024-04-24T23:11:04.343040+0000","last_clean":"2024-04-24T23:11:04.343040+0000","last_became_active":"2024-04-24T23:11:00.306929+0000","last_became_peered":"2024-04-24T23:11:00.306929+0000","last_unstale":"2024-04-24T23:11:04.343040+0000","last_undegraded":"2024-04-24T23:11:04.343040+0000","last_fullsized":"2024-04-24T23:11:04.343040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:19.234327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.215606+0000","last_change":"2024-04-24T23:11:01.214868+0000","last_active":"2024-04-24T23:11:01.215606+0000","last_peered":"2024-04-24T23:11:01.215606+0000","last_clean":"2024-04-24T23:11:01.215606+0000","last_became_active":"2024-04-24T23:11:01.211366+0000","last_became_peered":"2024-04-24T23:11:01.211366+0000","last_unstale":"2024-04-24T23:11:01.215606+0000","last_undegraded":"2024-04-24T23:11:01.215606+0000","last_fullsized":"2024-04-24T23:11:01.215606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:18:22.898831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.193479+0000","last_change":"2024-04-24T23:11:00.302050+0000","last_active":"2024-04-24T23:11:01.193479+0000","last_peered":"2024-04-24T23:11:01.193479+0000","last_clean":"2024-04-24T23:11:01.193479+0000","last_became_active":"2024-04-24T23:11:00.301740+0000","last_became_peered":"2024-04-24T23:11:00.301740+0000","last_unstale":"2024-04-24T23:11:01.193479+0000","last_undegraded":"2024-04-24T23:11:01.193479+0000","last_fullsized":"2024-04-24T23:11:01.193479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:34.994687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196485+0000","last_change":"2024-04-24T23:11:01.211470+0000","last_active":"2024-04-24T23:11:02.196485+0000","last_peered":"2024-04-24T23:11:02.196485+0000","last_clean":"2024-04-24T23:11:02.196485+0000","last_became_active":"2024-04-24T23:11:01.209482+0000","last_became_peered":"2024-04-24T23:11:01.209482+0000","last_unstale":"2024-04-24T23:11:02.196485+0000","last_undegraded":"2024-04-24T23:11:02.196485+0000","last_fullsized":"2024-04-24T23:11:02.196485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:01.407263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.199204+0000","last_change":"2024-04-24T23:11:00.305671+0000","last_active":"2024-04-24T23:11:01.199204+0000","last_peered":"2024-04-24T23:11:01.199204+0000","last_clean":"2024-04-24T23:11:01.199204+0000","last_became_active":"2024-04-24T23:11:00.303016+0000","last_became_peered":"2024-04-24T23:11:00.303016+0000","last_unstale":"2024-04-24T23:11:01.199204+0000","last_undegraded":"2024-04-24T23:11:01.199204+0000","last_fullsized":"2024-04-24T23:11:01.199204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:33.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618791+0000","last_change":"2024-04-24T23:11:01.206269+0000","last_active":"2024-04-24T23:11:02.618791+0000","last_peered":"2024-04-24T23:11:02.618791+0000","last_clean":"2024-04-24T23:11:02.618791+0000","last_became_active":"2024-04-24T23:11:01.205941+0000","last_became_peered":"2024-04-24T23:11:01.205941+0000","last_unstale":"2024-04-24T23:11:02.618791+0000","last_undegraded":"2024-04-24T23:11:02.618791+0000","last_fullsized":"2024-04-24T23:11:02.618791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:28.955247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:03.334458+0000","last_change":"2024-04-24T23:11:00.306071+0000","last_active":"2024-04-24T23:11:03.334458+0000","last_peered":"2024-04-24T23:11:03.334458+0000","last_clean":"2024-04-24T23:11:03.334458+0000","last_became_active":"2024-04-24T23:11:00.305751+0000","last_became_peered":"2024-04-24T23:11:00.305751+0000","last_unstale":"2024-04-24T23:11:03.334458+0000","last_undegraded":"2024-04-24T23:11:03.334458+0000","last_fullsized":"2024-04-24T23:11:03.334458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:30.675323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.222931+0000","last_change":"2024-04-24T23:11:01.222118+0000","last_active":"2024-04-24T23:11:01.222931+0000","last_peered":"2024-04-24T23:11:01.222931+0000","last_clean":"2024-04-24T23:11:01.222931+0000","last_became_active":"2024-04-24T23:11:01.221830+0000","last_became_peered":"2024-04-24T23:11:01.221830+0000","last_unstale":"2024-04-24T23:11:01.222931+0000","last_undegraded":"2024-04-24T23:11:01.222931+0000","last_fullsized":"2024-04-24T23:11:01.222931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:50.538146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618140+0000","last_change":"2024-04-24T23:11:00.308662+0000","last_active":"2024-04-24T23:11:02.618140+0000","last_peered":"2024-04-24T23:11:02.618140+0000","last_clean":"2024-04-24T23:11:02.618140+0000","last_became_active":"2024-04-24T23:11:00.308355+0000","last_became_peered":"2024-04-24T23:11:00.308355+0000","last_unstale":"2024-04-24T23:11:02.618140+0000","last_undegraded":"2024-04-24T23:11:02.618140+0000","last_fullsized":"2024-04-24T23:11:02.618140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:15.228277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618155+0000","last_change":"2024-04-24T23:11:01.210740+0000","last_active":"2024-04-24T23:11:02.618155+0000","last_peered":"2024-04-24T23:11:02.618155+0000","last_clean":"2024-04-24T23:11:02.618155+0000","last_became_active":"2024-04-24T23:11:01.210188+0000","last_became_peered":"2024-04-24T23:11:01.210188+0000","last_unstale":"2024-04-24T23:11:02.618155+0000","last_undegraded":"2024-04-24T23:11:02.618155+0000","last_fullsized":"2024-04-24T23:11:02.618155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:17.821343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.303043+0000","last_change":"2024-04-24T23:11:00.308114+0000","last_active":"2024-04-24T23:11:01.303043+0000","last_peered":"2024-04-24T23:11:01.303043+0000","last_clean":"2024-04-24T23:11:01.303043+0000","last_became_active":"2024-04-24T23:11:00.306632+0000","last_became_peered":"2024-04-24T23:11:00.306632+0000","last_unstale":"2024-04-24T23:11:01.303043+0000","last_undegraded":"2024-04-24T23:11:01.303043+0000","last_fullsized":"2024-04-24T23:11:01.303043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:19.298736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.221274+0000","last_change":"2024-04-24T23:11:01.219789+0000","last_active":"2024-04-24T23:11:01.221274+0000","last_peered":"2024-04-24T23:11:01.221274+0000","last_clean":"2024-04-24T23:11:01.221274+0000","last_became_active":"2024-04-24T23:11:01.219498+0000","last_became_peered":"2024-04-24T23:11:01.219498+0000","last_unstale":"2024-04-24T23:11:01.221274+0000","last_undegraded":"2024-04-24T23:11:01.221274+0000","last_fullsized":"2024-04-24T23:11:01.221274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:49.970335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:05.203634+0000","last_change":"2024-04-24T23:11:00.308847+0000","last_active":"2024-04-24T23:11:05.203634+0000","last_peered":"2024-04-24T23:11:05.203634+0000","last_clean":"2024-04-24T23:11:05.203634+0000","last_became_active":"2024-04-24T23:11:00.308493+0000","last_became_peered":"2024-04-24T23:11:00.308493+0000","last_unstale":"2024-04-24T23:11:05.203634+0000","last_undegraded":"2024-04-24T23:11:05.203634+0000","last_fullsized":"2024-04-24T23:11:05.203634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:24.354679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618728+0000","last_change":"2024-04-24T23:11:01.204997+0000","last_active":"2024-04-24T23:11:02.618728+0000","last_peered":"2024-04-24T23:11:02.618728+0000","last_clean":"2024-04-24T23:11:02.618728+0000","last_became_active":"2024-04-24T23:11:01.204645+0000","last_became_peered":"2024-04-24T23:11:01.204645+0000","last_unstale":"2024-04-24T23:11:02.618728+0000","last_undegraded":"2024-04-24T23:11:02.618728+0000","last_fullsized":"2024-04-24T23:11:02.618728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:07.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.342798+0000","last_change":"2024-04-24T23:11:00.302465+0000","last_active":"2024-04-24T23:11:04.342798+0000","last_peered":"2024-04-24T23:11:04.342798+0000","last_clean":"2024-04-24T23:11:04.342798+0000","last_became_active":"2024-04-24T23:11:00.302162+0000","last_became_peered":"2024-04-24T23:11:00.302162+0000","last_unstale":"2024-04-24T23:11:04.342798+0000","last_undegraded":"2024-04-24T23:11:04.342798+0000","last_fullsized":"2024-04-24T23:11:04.342798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:51.352872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.209883+0000","last_change":"2024-04-24T23:11:01.206588+0000","last_active":"2024-04-24T23:11:01.209883+0000","last_peered":"2024-04-24T23:11:01.209883+0000","last_clean":"2024-04-24T23:11:01.209883+0000","last_became_active":"2024-04-24T23:11:01.206258+0000","last_became_peered":"2024-04-24T23:11:01.206258+0000","last_unstale":"2024-04-24T23:11:01.209883+0000","last_undegraded":"2024-04-24T23:11:01.209883+0000","last_fullsized":"2024-04-24T23:11:01.209883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:44.917402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.198145+0000","last_change":"2024-04-24T23:11:00.305179+0000","last_active":"2024-04-24T23:11:01.198145+0000","last_peered":"2024-04-24T23:11:01.198145+0000","last_clean":"2024-04-24T23:11:01.198145+0000","last_became_active":"2024-04-24T23:11:00.302951+0000","last_became_peered":"2024-04-24T23:11:00.302951+0000","last_unstale":"2024-04-24T23:11:01.198145+0000","last_undegraded":"2024-04-24T23:11:01.198145+0000","last_fullsized":"2024-04-24T23:11:01.198145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:33.278498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618416+0000","last_change":"2024-04-24T23:11:01.211062+0000","last_active":"2024-04-24T23:11:02.618416+0000","last_peered":"2024-04-24T23:11:02.618416+0000","last_clean":"2024-04-24T23:11:02.618416+0000","last_became_active":"2024-04-24T23:11:01.210752+0000","last_became_peered":"2024-04-24T23:11:01.210752+0000","last_unstale":"2024-04-24T23:11:02.618416+0000","last_undegraded":"2024-04-24T23:11:02.618416+0000","last_fullsized":"2024-04-24T23:11:02.618416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:54.790212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.199412+0000","last_change":"2024-04-24T23:11:00.305690+0000","last_active":"2024-04-24T23:11:01.199412+0000","last_peered":"2024-04-24T23:11:01.199412+0000","last_clean":"2024-04-24T23:11:01.199412+0000","last_became_active":"2024-04-24T23:11:00.305386+0000","last_became_peered":"2024-04-24T23:11:00.305386+0000","last_unstale":"2024-04-24T23:11:01.199412+0000","last_undegraded":"2024-04-24T23:11:01.199412+0000","last_fullsized":"2024-04-24T23:11:01.199412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:39.898085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617802+0000","last_change":"2024-04-24T23:11:01.208916+0000","last_active":"2024-04-24T23:11:02.617802+0000","last_peered":"2024-04-24T23:11:02.617802+0000","last_clean":"2024-04-24T23:11:02.617802+0000","last_became_active":"2024-04-24T23:11:01.208593+0000","last_became_peered":"2024-04-24T23:11:01.208593+0000","last_unstale":"2024-04-24T23:11:02.617802+0000","last_undegraded":"2024-04-24T23:11:02.617802+0000","last_fullsized":"2024-04-24T23:11:02.617802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:58.442365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618001+0000","last_change":"2024-04-24T23:11:00.309608+0000","last_active":"2024-04-24T23:11:02.618001+0000","last_peered":"2024-04-24T23:11:02.618001+0000","last_clean":"2024-04-24T23:11:02.618001+0000","last_became_active":"2024-04-24T23:11:00.309310+0000","last_became_peered":"2024-04-24T23:11:00.309310+0000","last_unstale":"2024-04-24T23:11:02.618001+0000","last_undegraded":"2024-04-24T23:11:02.618001+0000","last_fullsized":"2024-04-24T23:11:02.618001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:23.330558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.213567+0000","last_change":"2024-04-24T23:11:01.212376+0000","last_active":"2024-04-24T23:11:01.213567+0000","last_peered":"2024-04-24T23:11:01.213567+0000","last_clean":"2024-04-24T23:11:01.213567+0000","last_became_active":"2024-04-24T23:11:01.212043+0000","last_became_peered":"2024-04-24T23:11:01.212043+0000","last_unstale":"2024-04-24T23:11:01.213567+0000","last_undegraded":"2024-04-24T23:11:01.213567+0000","last_fullsized":"2024-04-24T23:11:01.213567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:16:22.086023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618696+0000","last_change":"2024-04-24T23:11:00.305483+0000","last_active":"2024-04-24T23:11:02.618696+0000","last_peered":"2024-04-24T23:11:02.618696+0000","last_clean":"2024-04-24T23:11:02.618696+0000","last_became_active":"2024-04-24T23:11:00.305186+0000","last_became_peered":"2024-04-24T23:11:00.305186+0000","last_unstale":"2024-04-24T23:11:02.618696+0000","last_undegraded":"2024-04-24T23:11:02.618696+0000","last_fullsized":"2024-04-24T23:11:02.618696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:55.633200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618628+0000","last_change":"2024-04-24T23:11:01.210803+0000","last_active":"2024-04-24T23:11:02.618628+0000","last_peered":"2024-04-24T23:11:02.618628+0000","last_clean":"2024-04-24T23:11:02.618628+0000","last_became_active":"2024-04-24T23:11:01.210154+0000","last_became_peered":"2024-04-24T23:11:01.210154+0000","last_unstale":"2024-04-24T23:11:02.618628+0000","last_undegraded":"2024-04-24T23:11:02.618628+0000","last_fullsized":"2024-04-24T23:11:02.618628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:08.106252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618294+0000","last_change":"2024-04-24T23:11:00.305518+0000","last_active":"2024-04-24T23:11:02.618294+0000","last_peered":"2024-04-24T23:11:02.618294+0000","last_clean":"2024-04-24T23:11:02.618294+0000","last_became_active":"2024-04-24T23:11:00.305192+0000","last_became_peered":"2024-04-24T23:11:00.305192+0000","last_unstale":"2024-04-24T23:11:02.618294+0000","last_undegraded":"2024-04-24T23:11:02.618294+0000","last_fullsized":"2024-04-24T23:11:02.618294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:43:36.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618846+0000","last_change":"2024-04-24T23:11:01.207723+0000","last_active":"2024-04-24T23:11:02.618846+0000","last_peered":"2024-04-24T23:11:02.618846+0000","last_clean":"2024-04-24T23:11:02.618846+0000","last_became_active":"2024-04-24T23:11:01.207431+0000","last_became_peered":"2024-04-24T23:11:01.207431+0000","last_unstale":"2024-04-24T23:11:02.618846+0000","last_undegraded":"2024-04-24T23:11:02.618846+0000","last_fullsized":"2024-04-24T23:11:02.618846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:06.883404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618740+0000","last_change":"2024-04-24T23:11:00.310700+0000","last_active":"2024-04-24T23:11:02.618740+0000","last_peered":"2024-04-24T23:11:02.618740+0000","last_clean":"2024-04-24T23:11:02.618740+0000","last_became_active":"2024-04-24T23:11:00.310216+0000","last_became_peered":"2024-04-24T23:11:00.310216+0000","last_unstale":"2024-04-24T23:11:02.618740+0000","last_undegraded":"2024-04-24T23:11:02.618740+0000","last_fullsized":"2024-04-24T23:11:02.618740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:25.326695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618679+0000","last_change":"2024-04-24T23:11:01.214385+0000","last_active":"2024-04-24T23:11:02.618679+0000","last_peered":"2024-04-24T23:11:02.618679+0000","last_clean":"2024-04-24T23:11:02.618679+0000","last_became_active":"2024-04-24T23:11:01.214090+0000","last_became_peered":"2024-04-24T23:11:01.214090+0000","last_unstale":"2024-04-24T23:11:02.618679+0000","last_undegraded":"2024-04-24T23:11:02.618679+0000","last_fullsized":"2024-04-24T23:11:02.618679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:57.024833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.211107+0000","last_change":"2024-04-24T23:11:01.209826+0000","last_active":"2024-04-24T23:11:01.211107+0000","last_peered":"2024-04-24T23:11:01.211107+0000","last_clean":"2024-04-24T23:11:01.211107+0000","last_became_active":"2024-04-24T23:11:01.209432+0000","last_became_peered":"2024-04-24T23:11:01.209432+0000","last_unstale":"2024-04-24T23:11:01.211107+0000","last_undegraded":"2024-04-24T23:11:01.211107+0000","last_fullsized":"2024-04-24T23:11:01.211107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:43.176698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.193237+0000","last_change":"2024-04-24T23:11:00.305349+0000","last_active":"2024-04-24T23:11:01.193237+0000","last_peered":"2024-04-24T23:11:01.193237+0000","last_clean":"2024-04-24T23:11:01.193237+0000","last_became_active":"2024-04-24T23:11:00.305046+0000","last_became_peered":"2024-04-24T23:11:00.305046+0000","last_unstale":"2024-04-24T23:11:01.193237+0000","last_undegraded":"2024-04-24T23:11:01.193237+0000","last_fullsized":"2024-04-24T23:11:01.193237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:56.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196882+0000","last_change":"2024-04-24T23:11:01.216209+0000","last_active":"2024-04-24T23:11:02.196882+0000","last_peered":"2024-04-24T23:11:02.196882+0000","last_clean":"2024-04-24T23:11:02.196882+0000","last_became_active":"2024-04-24T23:11:01.215899+0000","last_became_peered":"2024-04-24T23:11:01.215899+0000","last_unstale":"2024-04-24T23:11:02.196882+0000","last_undegraded":"2024-04-24T23:11:02.196882+0000","last_fullsized":"2024-04-24T23:11:02.196882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:09.404193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.199588+0000","last_change":"2024-04-24T23:11:00.303160+0000","last_active":"2024-04-24T23:11:01.199588+0000","last_peered":"2024-04-24T23:11:01.199588+0000","last_clean":"2024-04-24T23:11:01.199588+0000","last_became_active":"2024-04-24T23:11:00.302781+0000","last_became_peered":"2024-04-24T23:11:00.302781+0000","last_unstale":"2024-04-24T23:11:01.199588+0000","last_undegraded":"2024-04-24T23:11:01.199588+0000","last_fullsized":"2024-04-24T23:11:01.199588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:42.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617863+0000","last_change":"2024-04-24T23:09:43.738402+0000","last_active":"2024-04-24T23:11:02.617863+0000","last_peered":"2024-04-24T23:11:02.617863+0000","last_clean":"2024-04-24T23:11:02.617863+0000","last_became_active":"2024-04-24T23:09:43.737979+0000","last_became_peered":"2024-04-24T23:09:43.737979+0000","last_unstale":"2024-04-24T23:11:02.617863+0000","last_undegraded":"2024-04-24T23:11:02.617863+0000","last_fullsized":"2024-04-24T23:11:02.617863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:16.456818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:16.456818+0000","last_clean_scrub_stamp":"2024-04-24T23:06:16.456818+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:52.449055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.209041+0000","last_change":"2024-04-24T23:11:01.203542+0000","last_active":"2024-04-24T23:11:01.209041+0000","last_peered":"2024-04-24T23:11:01.209041+0000","last_clean":"2024-04-24T23:11:01.209041+0000","last_became_active":"2024-04-24T23:11:01.203111+0000","last_became_peered":"2024-04-24T23:11:01.203111+0000","last_unstale":"2024-04-24T23:11:01.209041+0000","last_undegraded":"2024-04-24T23:11:01.209041+0000","last_fullsized":"2024-04-24T23:11:01.209041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:36.020377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618100+0000","last_change":"2024-04-24T23:11:00.304542+0000","last_active":"2024-04-24T23:11:02.618100+0000","last_peered":"2024-04-24T23:11:02.618100+0000","last_clean":"2024-04-24T23:11:02.618100+0000","last_became_active":"2024-04-24T23:11:00.304234+0000","last_became_peered":"2024-04-24T23:11:00.304234+0000","last_unstale":"2024-04-24T23:11:02.618100+0000","last_undegraded":"2024-04-24T23:11:02.618100+0000","last_fullsized":"2024-04-24T23:11:02.618100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:18.318876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618696+0000","last_change":"2024-04-24T23:11:00.310962+0000","last_active":"2024-04-24T23:11:02.618696+0000","last_peered":"2024-04-24T23:11:02.618696+0000","last_clean":"2024-04-24T23:11:02.618696+0000","last_became_active":"2024-04-24T23:11:00.310648+0000","last_became_peered":"2024-04-24T23:11:00.310648+0000","last_unstale":"2024-04-24T23:11:02.618696+0000","last_undegraded":"2024-04-24T23:11:02.618696+0000","last_fullsized":"2024-04-24T23:11:02.618696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:01.693517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.207975+0000","last_change":"2024-04-24T23:11:01.200293+0000","last_active":"2024-04-24T23:11:01.207975+0000","last_peered":"2024-04-24T23:11:01.207975+0000","last_clean":"2024-04-24T23:11:01.207975+0000","last_became_active":"2024-04-24T23:11:01.199945+0000","last_became_peered":"2024-04-24T23:11:01.199945+0000","last_unstale":"2024-04-24T23:11:01.207975+0000","last_undegraded":"2024-04-24T23:11:01.207975+0000","last_fullsized":"2024-04-24T23:11:01.207975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:05.561591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.343471+0000","last_change":"2024-04-24T23:11:00.307467+0000","last_active":"2024-04-24T23:11:04.343471+0000","last_peered":"2024-04-24T23:11:04.343471+0000","last_clean":"2024-04-24T23:11:04.343471+0000","last_became_active":"2024-04-24T23:11:00.307147+0000","last_became_peered":"2024-04-24T23:11:00.307147+0000","last_unstale":"2024-04-24T23:11:04.343471+0000","last_undegraded":"2024-04-24T23:11:04.343471+0000","last_fullsized":"2024-04-24T23:11:04.343471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:42.955410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.197072+0000","last_change":"2024-04-24T23:11:01.210541+0000","last_active":"2024-04-24T23:11:02.197072+0000","last_peered":"2024-04-24T23:11:02.197072+0000","last_clean":"2024-04-24T23:11:02.197072+0000","last_became_active":"2024-04-24T23:11:01.210187+0000","last_became_peered":"2024-04-24T23:11:01.210187+0000","last_unstale":"2024-04-24T23:11:02.197072+0000","last_undegraded":"2024-04-24T23:11:02.197072+0000","last_fullsized":"2024-04-24T23:11:02.197072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:45.138001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.300387+0000","last_change":"2024-04-24T23:11:00.308782+0000","last_active":"2024-04-24T23:11:01.300387+0000","last_peered":"2024-04-24T23:11:01.300387+0000","last_clean":"2024-04-24T23:11:01.300387+0000","last_became_active":"2024-04-24T23:11:00.308269+0000","last_became_peered":"2024-04-24T23:11:00.308269+0000","last_unstale":"2024-04-24T23:11:01.300387+0000","last_undegraded":"2024-04-24T23:11:01.300387+0000","last_fullsized":"2024-04-24T23:11:01.300387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:05.089037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617990+0000","last_change":"2024-04-24T23:11:01.214723+0000","last_active":"2024-04-24T23:11:02.617990+0000","last_peered":"2024-04-24T23:11:02.617990+0000","last_clean":"2024-04-24T23:11:02.617990+0000","last_became_active":"2024-04-24T23:11:01.213570+0000","last_became_peered":"2024-04-24T23:11:01.213570+0000","last_unstale":"2024-04-24T23:11:02.617990+0000","last_undegraded":"2024-04-24T23:11:02.617990+0000","last_fullsized":"2024-04-24T23:11:02.617990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:52.258640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.208683+0000","last_change":"2024-04-24T23:11:00.307522+0000","last_active":"2024-04-24T23:11:01.208683+0000","last_peered":"2024-04-24T23:11:01.208683+0000","last_clean":"2024-04-24T23:11:01.208683+0000","last_became_active":"2024-04-24T23:11:00.306106+0000","last_became_peered":"2024-04-24T23:11:00.306106+0000","last_unstale":"2024-04-24T23:11:01.208683+0000","last_undegraded":"2024-04-24T23:11:01.208683+0000","last_fullsized":"2024-04-24T23:11:01.208683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:21.623996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618057+0000","last_change":"2024-04-24T23:11:01.207426+0000","last_active":"2024-04-24T23:11:02.618057+0000","last_peered":"2024-04-24T23:11:02.618057+0000","last_clean":"2024-04-24T23:11:02.618057+0000","last_became_active":"2024-04-24T23:11:01.207020+0000","last_became_peered":"2024-04-24T23:11:01.207020+0000","last_unstale":"2024-04-24T23:11:02.618057+0000","last_undegraded":"2024-04-24T23:11:02.618057+0000","last_fullsized":"2024-04-24T23:11:02.618057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:58.389998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:03.332255+0000","last_change":"2024-04-24T23:11:00.302686+0000","last_active":"2024-04-24T23:11:03.332255+0000","last_peered":"2024-04-24T23:11:03.332255+0000","last_clean":"2024-04-24T23:11:03.332255+0000","last_became_active":"2024-04-24T23:11:00.302362+0000","last_became_peered":"2024-04-24T23:11:00.302362+0000","last_unstale":"2024-04-24T23:11:03.332255+0000","last_undegraded":"2024-04-24T23:11:03.332255+0000","last_fullsized":"2024-04-24T23:11:03.332255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:30.402780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618174+0000","last_change":"2024-04-24T23:11:01.212413+0000","last_active":"2024-04-24T23:11:02.618174+0000","last_peered":"2024-04-24T23:11:02.618174+0000","last_clean":"2024-04-24T23:11:02.618174+0000","last_became_active":"2024-04-24T23:11:01.211975+0000","last_became_peered":"2024-04-24T23:11:01.211975+0000","last_unstale":"2024-04-24T23:11:02.618174+0000","last_undegraded":"2024-04-24T23:11:02.618174+0000","last_fullsized":"2024-04-24T23:11:02.618174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:56.129977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.315735+0000","last_change":"2024-04-24T23:11:00.306583+0000","last_active":"2024-04-24T23:11:02.315735+0000","last_peered":"2024-04-24T23:11:02.315735+0000","last_clean":"2024-04-24T23:11:02.315735+0000","last_became_active":"2024-04-24T23:11:00.306274+0000","last_became_peered":"2024-04-24T23:11:00.306274+0000","last_unstale":"2024-04-24T23:11:02.315735+0000","last_undegraded":"2024-04-24T23:11:02.315735+0000","last_fullsized":"2024-04-24T23:11:02.315735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:15:58.157736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196656+0000","last_change":"2024-04-24T23:11:01.214041+0000","last_active":"2024-04-24T23:11:02.196656+0000","last_peered":"2024-04-24T23:11:02.196656+0000","last_clean":"2024-04-24T23:11:02.196656+0000","last_became_active":"2024-04-24T23:11:01.213639+0000","last_became_peered":"2024-04-24T23:11:01.213639+0000","last_unstale":"2024-04-24T23:11:02.196656+0000","last_undegraded":"2024-04-24T23:11:02.196656+0000","last_fullsized":"2024-04-24T23:11:02.196656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:08.819111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618427+0000","last_change":"2024-04-24T23:11:00.303790+0000","last_active":"2024-04-24T23:11:02.618427+0000","last_peered":"2024-04-24T23:11:02.618427+0000","last_clean":"2024-04-24T23:11:02.618427+0000","last_became_active":"2024-04-24T23:11:00.303381+0000","last_became_peered":"2024-04-24T23:11:00.303381+0000","last_unstale":"2024-04-24T23:11:02.618427+0000","last_undegraded":"2024-04-24T23:11:02.618427+0000","last_fullsized":"2024-04-24T23:11:02.618427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:18.725608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618846+0000","last_change":"2024-04-24T23:11:01.219575+0000","last_active":"2024-04-24T23:11:02.618846+0000","last_peered":"2024-04-24T23:11:02.618846+0000","last_clean":"2024-04-24T23:11:02.618846+0000","last_became_active":"2024-04-24T23:11:01.218750+0000","last_became_peered":"2024-04-24T23:11:01.218750+0000","last_unstale":"2024-04-24T23:11:02.618846+0000","last_undegraded":"2024-04-24T23:11:02.618846+0000","last_fullsized":"2024-04-24T23:11:02.618846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:00.945493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.343007+0000","last_change":"2024-04-24T23:11:00.310468+0000","last_active":"2024-04-24T23:11:04.343007+0000","last_peered":"2024-04-24T23:11:04.343007+0000","last_clean":"2024-04-24T23:11:04.343007+0000","last_became_active":"2024-04-24T23:11:00.310176+0000","last_became_peered":"2024-04-24T23:11:00.310176+0000","last_unstale":"2024-04-24T23:11:04.343007+0000","last_undegraded":"2024-04-24T23:11:04.343007+0000","last_fullsized":"2024-04-24T23:11:04.343007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:25.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618709+0000","last_change":"2024-04-24T23:11:01.219593+0000","last_active":"2024-04-24T23:11:02.618709+0000","last_peered":"2024-04-24T23:11:02.618709+0000","last_clean":"2024-04-24T23:11:02.618709+0000","last_became_active":"2024-04-24T23:11:01.219247+0000","last_became_peered":"2024-04-24T23:11:01.219247+0000","last_unstale":"2024-04-24T23:11:02.618709+0000","last_undegraded":"2024-04-24T23:11:02.618709+0000","last_fullsized":"2024-04-24T23:11:02.618709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:12.605797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.300746+0000","last_change":"2024-04-24T23:11:00.305124+0000","last_active":"2024-04-24T23:11:01.300746+0000","last_peered":"2024-04-24T23:11:01.300746+0000","last_clean":"2024-04-24T23:11:01.300746+0000","last_became_active":"2024-04-24T23:11:00.301468+0000","last_became_peered":"2024-04-24T23:11:00.301468+0000","last_unstale":"2024-04-24T23:11:01.300746+0000","last_undegraded":"2024-04-24T23:11:01.300746+0000","last_fullsized":"2024-04-24T23:11:01.300746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:58.451956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618010+0000","last_change":"2024-04-24T23:11:01.209967+0000","last_active":"2024-04-24T23:11:02.618010+0000","last_peered":"2024-04-24T23:11:02.618010+0000","last_clean":"2024-04-24T23:11:02.618010+0000","last_became_active":"2024-04-24T23:11:01.209607+0000","last_became_peered":"2024-04-24T23:11:01.209607+0000","last_unstale":"2024-04-24T23:11:02.618010+0000","last_undegraded":"2024-04-24T23:11:02.618010+0000","last_fullsized":"2024-04-24T23:11:02.618010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:02.840211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.302788+0000","last_change":"2024-04-24T23:11:00.308200+0000","last_active":"2024-04-24T23:11:01.302788+0000","last_peered":"2024-04-24T23:11:01.302788+0000","last_clean":"2024-04-24T23:11:01.302788+0000","last_became_active":"2024-04-24T23:11:00.306369+0000","last_became_peered":"2024-04-24T23:11:00.306369+0000","last_unstale":"2024-04-24T23:11:01.302788+0000","last_undegraded":"2024-04-24T23:11:01.302788+0000","last_fullsized":"2024-04-24T23:11:01.302788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:24.462361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618700+0000","last_change":"2024-04-24T23:11:01.207706+0000","last_active":"2024-04-24T23:11:02.618700+0000","last_peered":"2024-04-24T23:11:02.618700+0000","last_clean":"2024-04-24T23:11:02.618700+0000","last_became_active":"2024-04-24T23:11:01.207357+0000","last_became_peered":"2024-04-24T23:11:01.207357+0000","last_unstale":"2024-04-24T23:11:02.618700+0000","last_undegraded":"2024-04-24T23:11:02.618700+0000","last_fullsized":"2024-04-24T23:11:02.618700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:46.785844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618382+0000","last_change":"2024-04-24T23:11:01.214886+0000","last_active":"2024-04-24T23:11:02.618382+0000","last_peered":"2024-04-24T23:11:02.618382+0000","last_clean":"2024-04-24T23:11:02.618382+0000","last_became_active":"2024-04-24T23:11:01.213671+0000","last_became_peered":"2024-04-24T23:11:01.213671+0000","last_unstale":"2024-04-24T23:11:02.618382+0000","last_undegraded":"2024-04-24T23:11:02.618382+0000","last_fullsized":"2024-04-24T23:11:02.618382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:44.777437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.316051+0000","last_change":"2024-04-24T23:11:01.203578+0000","last_active":"2024-04-24T23:11:02.316051+0000","last_peered":"2024-04-24T23:11:02.316051+0000","last_clean":"2024-04-24T23:11:02.316051+0000","last_became_active":"2024-04-24T23:11:01.203250+0000","last_became_peered":"2024-04-24T23:11:01.203250+0000","last_unstale":"2024-04-24T23:11:02.316051+0000","last_undegraded":"2024-04-24T23:11:02.316051+0000","last_fullsized":"2024-04-24T23:11:02.316051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:39.050149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.316102+0000","last_change":"2024-04-24T23:11:01.209641+0000","last_active":"2024-04-24T23:11:02.316102+0000","last_peered":"2024-04-24T23:11:02.316102+0000","last_clean":"2024-04-24T23:11:02.316102+0000","last_became_active":"2024-04-24T23:11:01.209110+0000","last_became_peered":"2024-04-24T23:11:01.209110+0000","last_unstale":"2024-04-24T23:11:02.316102+0000","last_undegraded":"2024-04-24T23:11:02.316102+0000","last_fullsized":"2024-04-24T23:11:02.316102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:14.709926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617842+0000","last_change":"2024-04-24T23:11:01.213894+0000","last_active":"2024-04-24T23:11:02.617842+0000","last_peered":"2024-04-24T23:11:02.617842+0000","last_clean":"2024-04-24T23:11:02.617842+0000","last_became_active":"2024-04-24T23:11:01.213461+0000","last_became_peered":"2024-04-24T23:11:01.213461+0000","last_unstale":"2024-04-24T23:11:02.617842+0000","last_undegraded":"2024-04-24T23:11:02.617842+0000","last_fullsized":"2024-04-24T23:11:02.617842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:00.786413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.209228+0000","last_change":"2024-04-24T23:11:01.206556+0000","last_active":"2024-04-24T23:11:01.209228+0000","last_peered":"2024-04-24T23:11:01.209228+0000","last_clean":"2024-04-24T23:11:01.209228+0000","last_became_active":"2024-04-24T23:11:01.206178+0000","last_became_peered":"2024-04-24T23:11:01.206178+0000","last_unstale":"2024-04-24T23:11:01.209228+0000","last_undegraded":"2024-04-24T23:11:01.209228+0000","last_fullsized":"2024-04-24T23:11:01.209228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:00.175173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618585+0000","last_change":"2024-04-24T23:11:01.213174+0000","last_active":"2024-04-24T23:11:02.618585+0000","last_peered":"2024-04-24T23:11:02.618585+0000","last_clean":"2024-04-24T23:11:02.618585+0000","last_became_active":"2024-04-24T23:11:01.212815+0000","last_became_peered":"2024-04-24T23:11:01.212815+0000","last_unstale":"2024-04-24T23:11:02.618585+0000","last_undegraded":"2024-04-24T23:11:02.618585+0000","last_fullsized":"2024-04-24T23:11:02.618585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:09.281324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.316312+0000","last_change":"2024-04-24T23:11:01.215030+0000","last_active":"2024-04-24T23:11:02.316312+0000","last_peered":"2024-04-24T23:11:02.316312+0000","last_clean":"2024-04-24T23:11:02.316312+0000","last_became_active":"2024-04-24T23:11:01.214386+0000","last_became_peered":"2024-04-24T23:11:01.214386+0000","last_unstale":"2024-04-24T23:11:02.316312+0000","last_undegraded":"2024-04-24T23:11:02.316312+0000","last_fullsized":"2024-04-24T23:11:02.316312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:10.633565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196642+0000","last_change":"2024-04-24T23:11:01.217748+0000","last_active":"2024-04-24T23:11:02.196642+0000","last_peered":"2024-04-24T23:11:02.196642+0000","last_clean":"2024-04-24T23:11:02.196642+0000","last_became_active":"2024-04-24T23:11:01.217403+0000","last_became_peered":"2024-04-24T23:11:01.217403+0000","last_unstale":"2024-04-24T23:11:02.196642+0000","last_undegraded":"2024-04-24T23:11:02.196642+0000","last_fullsized":"2024-04-24T23:11:02.196642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:29.569179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.210197+0000","last_change":"2024-04-24T23:11:01.208894+0000","last_active":"2024-04-24T23:11:01.210197+0000","last_peered":"2024-04-24T23:11:01.210197+0000","last_clean":"2024-04-24T23:11:01.210197+0000","last_became_active":"2024-04-24T23:11:01.208576+0000","last_became_peered":"2024-04-24T23:11:01.208576+0000","last_unstale":"2024-04-24T23:11:01.210197+0000","last_undegraded":"2024-04-24T23:11:01.210197+0000","last_fullsized":"2024-04-24T23:11:01.210197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:55.852486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196521+0000","last_change":"2024-04-24T23:11:01.218561+0000","last_active":"2024-04-24T23:11:02.196521+0000","last_peered":"2024-04-24T23:11:02.196521+0000","last_clean":"2024-04-24T23:11:02.196521+0000","last_became_active":"2024-04-24T23:11:01.218261+0000","last_became_peered":"2024-04-24T23:11:01.218261+0000","last_unstale":"2024-04-24T23:11:02.196521+0000","last_undegraded":"2024-04-24T23:11:02.196521+0000","last_fullsized":"2024-04-24T23:11:02.196521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:49.677225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.216200+0000","last_change":"2024-04-24T23:11:01.214868+0000","last_active":"2024-04-24T23:11:01.216200+0000","last_peered":"2024-04-24T23:11:01.216200+0000","last_clean":"2024-04-24T23:11:01.216200+0000","last_became_active":"2024-04-24T23:11:01.214582+0000","last_became_peered":"2024-04-24T23:11:01.214582+0000","last_unstale":"2024-04-24T23:11:01.216200+0000","last_undegraded":"2024-04-24T23:11:01.216200+0000","last_fullsized":"2024-04-24T23:11:01.216200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:38.201838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.215938+0000","last_change":"2024-04-24T23:11:01.212348+0000","last_active":"2024-04-24T23:11:01.215938+0000","last_peered":"2024-04-24T23:11:01.215938+0000","last_clean":"2024-04-24T23:11:01.215938+0000","last_became_active":"2024-04-24T23:11:01.211802+0000","last_became_peered":"2024-04-24T23:11:01.211802+0000","last_unstale":"2024-04-24T23:11:01.215938+0000","last_undegraded":"2024-04-24T23:11:01.215938+0000","last_fullsized":"2024-04-24T23:11:01.215938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:12.552821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618276+0000","last_change":"2024-04-24T23:11:01.209834+0000","last_active":"2024-04-24T23:11:02.618276+0000","last_peered":"2024-04-24T23:11:02.618276+0000","last_clean":"2024-04-24T23:11:02.618276+0000","last_became_active":"2024-04-24T23:11:01.209476+0000","last_became_peered":"2024-04-24T23:11:01.209476+0000","last_unstale":"2024-04-24T23:11:02.618276+0000","last_undegraded":"2024-04-24T23:11:02.618276+0000","last_fullsized":"2024-04-24T23:11:02.618276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:20.036684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244416,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9772,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":81,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":24328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":913408,"data_stored":434445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1043564,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":443833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":435049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":1033965,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":434445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":425368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":434535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":445977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":443299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":425368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1024813,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:11:07.548 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:11:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:07 smithi161 ceph-mon[32709]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:07 smithi161 ceph-mon[32709]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:07 smithi161 ceph-mon[32709]: pgmap v295: 97 pgs: 97 active+clean; 586 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-24T23:11:07.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:07 smithi161 ceph-mon[32709]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:07.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:07 smithi161 ceph-mon[32709]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:11:07.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:07 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/368718556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:07 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3961857200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 -- 172.21.15.102:0/2021354494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8101790 msgr2=0x7f4ae8101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2021354494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8101790 0x7f4ae8101bb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009ca0 tx=0x7f4ae002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 -- 172.21.15.102:0/2021354494 shutdown_connections 2024-04-24T23:11:07.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2021354494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae8103340 0x7f4ae8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2021354494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae8102980 0x7f4ae8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2021354494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8101790 0x7f4ae8101bb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 -- 172.21.15.102:0/2021354494 >> 172.21.15.102:0/2021354494 conn(0x7f4ae80fcd20 msgr2=0x7f4ae80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 -- 172.21.15.102:0/2021354494 shutdown_connections 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 -- 172.21.15.102:0/2021354494 wait complete. 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 Processor -- start 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 -- start start 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae8101790 0x7f4ae81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8102980 0x7f4ae8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.679+0000 7f4aeed16640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae8103340 0x7f4ae819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4aeed16640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae810d310 con 0x7f4ae8102980 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4aeed16640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae810d190 con 0x7f4ae8101790 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4aeed16640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae810d490 con 0x7f4ae8103340 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8102980 0x7f4ae8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8102980 0x7f4ae8197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55712/0 (socket says 172.21.15.102:55712) 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae7fff640 1 -- 172.21.15.102:0/2476467699 learned_addr learned my addr 172.21.15.102:0/2476467699 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4aecab2640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae8101790 0x7f4ae81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae7fff640 1 -- 172.21.15.102:0/2476467699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae8103340 msgr2=0x7f4ae819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae7fff640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae8103340 0x7f4ae819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae7fff640 1 -- 172.21.15.102:0/2476467699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae8101790 msgr2=0x7f4ae81979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae7fff640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae8101790 0x7f4ae81979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae7fff640 1 -- 172.21.15.102:0/2476467699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae819f610 con 0x7f4ae8102980 2024-04-24T23:11:07.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae7fff640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8102980 0x7f4ae8197f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b4a0 tx=0x7f4ad800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae5ffb640 1 -- 172.21.15.102:0/2476467699 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8014070 con 0x7f4ae8102980 2024-04-24T23:11:07.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae5ffb640 1 -- 172.21.15.102:0/2476467699 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ad8010ac0 con 0x7f4ae8102980 2024-04-24T23:11:07.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae819f900 con 0x7f4ae8102980 2024-04-24T23:11:07.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4ae5ffb640 1 -- 172.21.15.102:0/2476467699 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8017c30 con 0x7f4ae8102980 2024-04-24T23:11:07.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.680+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae819fe50 con 0x7f4ae8102980 2024-04-24T23:11:07.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.681+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae8108b30 con 0x7f4ae8102980 2024-04-24T23:11:07.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.682+0000 7f4ae5ffb640 1 -- 172.21.15.102:0/2476467699 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4ad8017070 con 0x7f4ae8102980 2024-04-24T23:11:07.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.683+0000 7f4ae5ffb640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4ad40a2c20 0x7f4ad40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.683+0000 7f4aecab2640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4ad40a2c20 0x7f4ad40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.683+0000 7f4ae5ffb640 1 -- 172.21.15.102:0/2476467699 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4ad80ce690 con 0x7f4ae8102980 2024-04-24T23:11:07.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.683+0000 7f4aecab2640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4ad40a2c20 0x7f4ad40a50e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006010 tx=0x7f4ae0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.687+0000 7f4ae5ffb640 1 -- 172.21.15.102:0/2476467699 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4ad809b750 con 0x7f4ae8102980 2024-04-24T23:11:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:07 smithi183 ceph-mon[33137]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:07 smithi183 ceph-mon[33137]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:07 smithi183 ceph-mon[33137]: pgmap v295: 97 pgs: 97 active+clean; 586 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-24T23:11:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:07 smithi183 ceph-mon[33137]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:07 smithi183 ceph-mon[33137]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:11:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:07 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/368718556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:07 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3961857200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.859+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ae8061b80 con 0x7f4ad40a2c20 2024-04-24T23:11:07.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.866+0000 7f4ae5ffb640 1 -- 172.21.15.102:0/2476467699 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7f4ae8061b80 con 0x7f4ad40a2c20 2024-04-24T23:11:07.871 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:11:07.872 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-24T23:11:07.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4ad40a2c20 msgr2=0x7f4ad40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4ad40a2c20 0x7f4ad40a50e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006010 tx=0x7f4ae0000f40 comp rx=0 tx=0).stop 2024-04-24T23:11:07.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8102980 msgr2=0x7f4ae8197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8102980 0x7f4ae8197f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b4a0 tx=0x7f4ad800b960 comp rx=0 tx=0).stop 2024-04-24T23:11:07.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 shutdown_connections 2024-04-24T23:11:07.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f4ad40a2c20 0x7f4ad40a50e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae8103340 0x7f4ae819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ae8102980 0x7f4ae8197f30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 --2- 172.21.15.102:0/2476467699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae8101790 0x7f4ae81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 >> 172.21.15.102:0/2476467699 conn(0x7f4ae80fcd20 msgr2=0x7f4ae8105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 shutdown_connections 2024-04-24T23:11:07.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:07.869+0000 7f4aeed16640 1 -- 172.21.15.102:0/2476467699 wait complete. 2024-04-24T23:11:07.882 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-04-24T23:11:06.218153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600140,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":144,"num_read_kb":126,"num_write":262,"num_write_kb":2963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356072,"kb_used_data":11020,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585752,"statfs":{"total":1151940427776,"available":1151575810048,"internally_reserved":0,"allocated":11284480,"data_stored":7024661,"data_compressed":106857,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9772,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":81,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001364"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618869+0000","last_change":"2024-04-24T23:11:01.217761+0000","last_active":"2024-04-24T23:11:02.618869+0000","last_peered":"2024-04-24T23:11:02.618869+0000","last_clean":"2024-04-24T23:11:02.618869+0000","last_became_active":"2024-04-24T23:11:01.217330+0000","last_became_peered":"2024-04-24T23:11:01.217330+0000","last_unstale":"2024-04-24T23:11:02.618869+0000","last_undegraded":"2024-04-24T23:11:02.618869+0000","last_fullsized":"2024-04-24T23:11:02.618869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:10.690940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617991+0000","last_change":"2024-04-24T23:11:01.206722+0000","last_active":"2024-04-24T23:11:02.617991+0000","last_peered":"2024-04-24T23:11:02.617991+0000","last_clean":"2024-04-24T23:11:02.617991+0000","last_became_active":"2024-04-24T23:11:01.205573+0000","last_became_peered":"2024-04-24T23:11:01.205573+0000","last_unstale":"2024-04-24T23:11:02.617991+0000","last_undegraded":"2024-04-24T23:11:02.617991+0000","last_fullsized":"2024-04-24T23:11:02.617991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:01.757792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617957+0000","last_change":"2024-04-24T23:11:01.213159+0000","last_active":"2024-04-24T23:11:02.617957+0000","last_peered":"2024-04-24T23:11:02.617957+0000","last_clean":"2024-04-24T23:11:02.617957+0000","last_became_active":"2024-04-24T23:11:01.212811+0000","last_became_peered":"2024-04-24T23:11:01.212811+0000","last_unstale":"2024-04-24T23:11:02.617957+0000","last_undegraded":"2024-04-24T23:11:02.617957+0000","last_fullsized":"2024-04-24T23:11:02.617957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:02.053206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.215153+0000","last_change":"2024-04-24T23:11:01.213023+0000","last_active":"2024-04-24T23:11:01.215153+0000","last_peered":"2024-04-24T23:11:01.215153+0000","last_clean":"2024-04-24T23:11:01.215153+0000","last_became_active":"2024-04-24T23:11:01.212689+0000","last_became_peered":"2024-04-24T23:11:01.212689+0000","last_unstale":"2024-04-24T23:11:01.215153+0000","last_undegraded":"2024-04-24T23:11:01.215153+0000","last_fullsized":"2024-04-24T23:11:01.215153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:00.942199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618302+0000","last_change":"2024-04-24T23:11:01.210845+0000","last_active":"2024-04-24T23:11:02.618302+0000","last_peered":"2024-04-24T23:11:02.618302+0000","last_clean":"2024-04-24T23:11:02.618302+0000","last_became_active":"2024-04-24T23:11:01.210389+0000","last_became_peered":"2024-04-24T23:11:01.210389+0000","last_unstale":"2024-04-24T23:11:02.618302+0000","last_undegraded":"2024-04-24T23:11:02.618302+0000","last_fullsized":"2024-04-24T23:11:02.618302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:03.946294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618518+0000","last_change":"2024-04-24T23:11:01.203082+0000","last_active":"2024-04-24T23:11:02.618518+0000","last_peered":"2024-04-24T23:11:02.618518+0000","last_clean":"2024-04-24T23:11:02.618518+0000","last_became_active":"2024-04-24T23:11:01.202605+0000","last_became_peered":"2024-04-24T23:11:01.202605+0000","last_unstale":"2024-04-24T23:11:02.618518+0000","last_undegraded":"2024-04-24T23:11:02.618518+0000","last_fullsized":"2024-04-24T23:11:02.618518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:29.248361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618138+0000","last_change":"2024-04-24T23:11:01.213002+0000","last_active":"2024-04-24T23:11:02.618138+0000","last_peered":"2024-04-24T23:11:02.618138+0000","last_clean":"2024-04-24T23:11:02.618138+0000","last_became_active":"2024-04-24T23:11:01.212690+0000","last_became_peered":"2024-04-24T23:11:01.212690+0000","last_unstale":"2024-04-24T23:11:02.618138+0000","last_undegraded":"2024-04-24T23:11:02.618138+0000","last_fullsized":"2024-04-24T23:11:02.618138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:06.658975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196454+0000","last_change":"2024-04-24T23:11:01.213041+0000","last_active":"2024-04-24T23:11:02.196454+0000","last_peered":"2024-04-24T23:11:02.196454+0000","last_clean":"2024-04-24T23:11:02.196454+0000","last_became_active":"2024-04-24T23:11:01.212659+0000","last_became_peered":"2024-04-24T23:11:01.212659+0000","last_unstale":"2024-04-24T23:11:02.196454+0000","last_undegraded":"2024-04-24T23:11:02.196454+0000","last_fullsized":"2024-04-24T23:11:02.196454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:53.229204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.217998+0000","last_change":"2024-04-24T23:11:01.217638+0000","last_active":"2024-04-24T23:11:01.217998+0000","last_peered":"2024-04-24T23:11:01.217998+0000","last_clean":"2024-04-24T23:11:01.217998+0000","last_became_active":"2024-04-24T23:11:01.217288+0000","last_became_peered":"2024-04-24T23:11:01.217288+0000","last_unstale":"2024-04-24T23:11:01.217998+0000","last_undegraded":"2024-04-24T23:11:01.217998+0000","last_fullsized":"2024-04-24T23:11:01.217998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:04.113776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618647+0000","last_change":"2024-04-24T23:11:01.203188+0000","last_active":"2024-04-24T23:11:02.618647+0000","last_peered":"2024-04-24T23:11:02.618647+0000","last_clean":"2024-04-24T23:11:02.618647+0000","last_became_active":"2024-04-24T23:11:01.202763+0000","last_became_peered":"2024-04-24T23:11:01.202763+0000","last_unstale":"2024-04-24T23:11:02.618647+0000","last_undegraded":"2024-04-24T23:11:02.618647+0000","last_fullsized":"2024-04-24T23:11:02.618647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:04.414650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618545+0000","last_change":"2024-04-24T23:11:01.220641+0000","last_active":"2024-04-24T23:11:02.618545+0000","last_peered":"2024-04-24T23:11:02.618545+0000","last_clean":"2024-04-24T23:11:02.618545+0000","last_became_active":"2024-04-24T23:11:01.220344+0000","last_became_peered":"2024-04-24T23:11:01.220344+0000","last_unstale":"2024-04-24T23:11:02.618545+0000","last_undegraded":"2024-04-24T23:11:02.618545+0000","last_fullsized":"2024-04-24T23:11:02.618545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:44.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.210929+0000","last_change":"2024-04-24T23:11:01.209805+0000","last_active":"2024-04-24T23:11:01.210929+0000","last_peered":"2024-04-24T23:11:01.210929+0000","last_clean":"2024-04-24T23:11:01.210929+0000","last_became_active":"2024-04-24T23:11:01.209497+0000","last_became_peered":"2024-04-24T23:11:01.209497+0000","last_unstale":"2024-04-24T23:11:01.210929+0000","last_undegraded":"2024-04-24T23:11:01.210929+0000","last_fullsized":"2024-04-24T23:11:01.210929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:27.930210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.210893+0000","last_change":"2024-04-24T23:11:01.205763+0000","last_active":"2024-04-24T23:11:01.210893+0000","last_peered":"2024-04-24T23:11:01.210893+0000","last_clean":"2024-04-24T23:11:01.210893+0000","last_became_active":"2024-04-24T23:11:01.205402+0000","last_became_peered":"2024-04-24T23:11:01.205402+0000","last_unstale":"2024-04-24T23:11:01.210893+0000","last_undegraded":"2024-04-24T23:11:01.210893+0000","last_fullsized":"2024-04-24T23:11:01.210893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:27.816472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617492+0000","last_change":"2024-04-24T23:11:01.209466+0000","last_active":"2024-04-24T23:11:02.617492+0000","last_peered":"2024-04-24T23:11:02.617492+0000","last_clean":"2024-04-24T23:11:02.617492+0000","last_became_active":"2024-04-24T23:11:01.208946+0000","last_became_peered":"2024-04-24T23:11:01.208946+0000","last_unstale":"2024-04-24T23:11:02.617492+0000","last_undegraded":"2024-04-24T23:11:02.617492+0000","last_fullsized":"2024-04-24T23:11:02.617492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:07.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617512+0000","last_change":"2024-04-24T23:11:01.215170+0000","last_active":"2024-04-24T23:11:02.617512+0000","last_peered":"2024-04-24T23:11:02.617512+0000","last_clean":"2024-04-24T23:11:02.617512+0000","last_became_active":"2024-04-24T23:11:01.214867+0000","last_became_peered":"2024-04-24T23:11:01.214867+0000","last_unstale":"2024-04-24T23:11:02.617512+0000","last_undegraded":"2024-04-24T23:11:02.617512+0000","last_fullsized":"2024-04-24T23:11:02.617512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.830234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.217138+0000","last_change":"2024-04-24T23:11:01.216691+0000","last_active":"2024-04-24T23:11:01.217138+0000","last_peered":"2024-04-24T23:11:01.217138+0000","last_clean":"2024-04-24T23:11:01.217138+0000","last_became_active":"2024-04-24T23:11:01.216407+0000","last_became_peered":"2024-04-24T23:11:01.216407+0000","last_unstale":"2024-04-24T23:11:01.217138+0000","last_undegraded":"2024-04-24T23:11:01.217138+0000","last_fullsized":"2024-04-24T23:11:01.217138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:13.289291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618946+0000","last_change":"2024-04-24T23:11:01.217764+0000","last_active":"2024-04-24T23:11:02.618946+0000","last_peered":"2024-04-24T23:11:02.618946+0000","last_clean":"2024-04-24T23:11:02.618946+0000","last_became_active":"2024-04-24T23:11:01.217424+0000","last_became_peered":"2024-04-24T23:11:01.217424+0000","last_unstale":"2024-04-24T23:11:02.618946+0000","last_undegraded":"2024-04-24T23:11:02.618946+0000","last_fullsized":"2024-04-24T23:11:02.618946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:11.517317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618394+0000","last_change":"2024-04-24T23:11:01.214351+0000","last_active":"2024-04-24T23:11:02.618394+0000","last_peered":"2024-04-24T23:11:02.618394+0000","last_clean":"2024-04-24T23:11:02.618394+0000","last_became_active":"2024-04-24T23:11:01.214034+0000","last_became_peered":"2024-04-24T23:11:01.214034+0000","last_unstale":"2024-04-24T23:11:02.618394+0000","last_undegraded":"2024-04-24T23:11:02.618394+0000","last_fullsized":"2024-04-24T23:11:02.618394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:26.105888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.316062+0000","last_change":"2024-04-24T23:11:01.212836+0000","last_active":"2024-04-24T23:11:02.316062+0000","last_peered":"2024-04-24T23:11:02.316062+0000","last_clean":"2024-04-24T23:11:02.316062+0000","last_became_active":"2024-04-24T23:11:01.209223+0000","last_became_peered":"2024-04-24T23:11:01.209223+0000","last_unstale":"2024-04-24T23:11:02.316062+0000","last_undegraded":"2024-04-24T23:11:02.316062+0000","last_fullsized":"2024-04-24T23:11:02.316062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:40.100399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.343540+0000","last_change":"2024-04-24T23:11:00.302055+0000","last_active":"2024-04-24T23:11:04.343540+0000","last_peered":"2024-04-24T23:11:04.343540+0000","last_clean":"2024-04-24T23:11:04.343540+0000","last_became_active":"2024-04-24T23:11:00.301726+0000","last_became_peered":"2024-04-24T23:11:00.301726+0000","last_unstale":"2024-04-24T23:11:04.343540+0000","last_undegraded":"2024-04-24T23:11:04.343540+0000","last_fullsized":"2024-04-24T23:11:04.343540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:54.039099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196683+0000","last_change":"2024-04-24T23:11:01.219527+0000","last_active":"2024-04-24T23:11:02.196683+0000","last_peered":"2024-04-24T23:11:02.196683+0000","last_clean":"2024-04-24T23:11:02.196683+0000","last_became_active":"2024-04-24T23:11:01.219196+0000","last_became_peered":"2024-04-24T23:11:01.219196+0000","last_unstale":"2024-04-24T23:11:02.196683+0000","last_undegraded":"2024-04-24T23:11:02.196683+0000","last_fullsized":"2024-04-24T23:11:02.196683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:58.584402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196796+0000","last_change":"2024-04-24T23:11:00.309260+0000","last_active":"2024-04-24T23:11:02.196796+0000","last_peered":"2024-04-24T23:11:02.196796+0000","last_clean":"2024-04-24T23:11:02.196796+0000","last_became_active":"2024-04-24T23:11:00.308969+0000","last_became_peered":"2024-04-24T23:11:00.308969+0000","last_unstale":"2024-04-24T23:11:02.196796+0000","last_undegraded":"2024-04-24T23:11:02.196796+0000","last_fullsized":"2024-04-24T23:11:02.196796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:23.137469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618171+0000","last_change":"2024-04-24T23:11:01.216515+0000","last_active":"2024-04-24T23:11:02.618171+0000","last_peered":"2024-04-24T23:11:02.618171+0000","last_clean":"2024-04-24T23:11:02.618171+0000","last_became_active":"2024-04-24T23:11:01.216168+0000","last_became_peered":"2024-04-24T23:11:01.216168+0000","last_unstale":"2024-04-24T23:11:02.618171+0000","last_undegraded":"2024-04-24T23:11:02.618171+0000","last_fullsized":"2024-04-24T23:11:02.618171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:05.229365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618850+0000","last_change":"2024-04-24T23:11:00.302558+0000","last_active":"2024-04-24T23:11:02.618850+0000","last_peered":"2024-04-24T23:11:02.618850+0000","last_clean":"2024-04-24T23:11:02.618850+0000","last_became_active":"2024-04-24T23:11:00.302162+0000","last_became_peered":"2024-04-24T23:11:00.302162+0000","last_unstale":"2024-04-24T23:11:02.618850+0000","last_undegraded":"2024-04-24T23:11:02.618850+0000","last_fullsized":"2024-04-24T23:11:02.618850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:52.623679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.222961+0000","last_change":"2024-04-24T23:11:01.222129+0000","last_active":"2024-04-24T23:11:01.222961+0000","last_peered":"2024-04-24T23:11:01.222961+0000","last_clean":"2024-04-24T23:11:01.222961+0000","last_became_active":"2024-04-24T23:11:01.221795+0000","last_became_peered":"2024-04-24T23:11:01.221795+0000","last_unstale":"2024-04-24T23:11:01.222961+0000","last_undegraded":"2024-04-24T23:11:01.222961+0000","last_fullsized":"2024-04-24T23:11:01.222961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:35.194854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618553+0000","last_change":"2024-04-24T23:11:00.308586+0000","last_active":"2024-04-24T23:11:02.618553+0000","last_peered":"2024-04-24T23:11:02.618553+0000","last_clean":"2024-04-24T23:11:02.618553+0000","last_became_active":"2024-04-24T23:11:00.305054+0000","last_became_peered":"2024-04-24T23:11:00.305054+0000","last_unstale":"2024-04-24T23:11:02.618553+0000","last_undegraded":"2024-04-24T23:11:02.618553+0000","last_fullsized":"2024-04-24T23:11:02.618553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:11.371829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.212468+0000","last_change":"2024-04-24T23:11:01.211868+0000","last_active":"2024-04-24T23:11:01.212468+0000","last_peered":"2024-04-24T23:11:01.212468+0000","last_clean":"2024-04-24T23:11:01.212468+0000","last_became_active":"2024-04-24T23:11:01.211504+0000","last_became_peered":"2024-04-24T23:11:01.211504+0000","last_unstale":"2024-04-24T23:11:01.212468+0000","last_undegraded":"2024-04-24T23:11:01.212468+0000","last_fullsized":"2024-04-24T23:11:01.212468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:47.974137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.302721+0000","last_change":"2024-04-24T23:11:00.302389+0000","last_active":"2024-04-24T23:11:01.302721+0000","last_peered":"2024-04-24T23:11:01.302721+0000","last_clean":"2024-04-24T23:11:01.302721+0000","last_became_active":"2024-04-24T23:11:00.302061+0000","last_became_peered":"2024-04-24T23:11:00.302061+0000","last_unstale":"2024-04-24T23:11:01.302721+0000","last_undegraded":"2024-04-24T23:11:01.302721+0000","last_fullsized":"2024-04-24T23:11:01.302721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:28.437391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618464+0000","last_change":"2024-04-24T23:11:01.202389+0000","last_active":"2024-04-24T23:11:02.618464+0000","last_peered":"2024-04-24T23:11:02.618464+0000","last_clean":"2024-04-24T23:11:02.618464+0000","last_became_active":"2024-04-24T23:11:01.201986+0000","last_became_peered":"2024-04-24T23:11:01.201986+0000","last_unstale":"2024-04-24T23:11:02.618464+0000","last_undegraded":"2024-04-24T23:11:02.618464+0000","last_fullsized":"2024-04-24T23:11:02.618464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:25.743284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.343040+0000","last_change":"2024-04-24T23:11:00.307278+0000","last_active":"2024-04-24T23:11:04.343040+0000","last_peered":"2024-04-24T23:11:04.343040+0000","last_clean":"2024-04-24T23:11:04.343040+0000","last_became_active":"2024-04-24T23:11:00.306929+0000","last_became_peered":"2024-04-24T23:11:00.306929+0000","last_unstale":"2024-04-24T23:11:04.343040+0000","last_undegraded":"2024-04-24T23:11:04.343040+0000","last_fullsized":"2024-04-24T23:11:04.343040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:19.234327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.215606+0000","last_change":"2024-04-24T23:11:01.214868+0000","last_active":"2024-04-24T23:11:01.215606+0000","last_peered":"2024-04-24T23:11:01.215606+0000","last_clean":"2024-04-24T23:11:01.215606+0000","last_became_active":"2024-04-24T23:11:01.211366+0000","last_became_peered":"2024-04-24T23:11:01.211366+0000","last_unstale":"2024-04-24T23:11:01.215606+0000","last_undegraded":"2024-04-24T23:11:01.215606+0000","last_fullsized":"2024-04-24T23:11:01.215606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:18:22.898831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.193479+0000","last_change":"2024-04-24T23:11:00.302050+0000","last_active":"2024-04-24T23:11:01.193479+0000","last_peered":"2024-04-24T23:11:01.193479+0000","last_clean":"2024-04-24T23:11:01.193479+0000","last_became_active":"2024-04-24T23:11:00.301740+0000","last_became_peered":"2024-04-24T23:11:00.301740+0000","last_unstale":"2024-04-24T23:11:01.193479+0000","last_undegraded":"2024-04-24T23:11:01.193479+0000","last_fullsized":"2024-04-24T23:11:01.193479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:34.994687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196485+0000","last_change":"2024-04-24T23:11:01.211470+0000","last_active":"2024-04-24T23:11:02.196485+0000","last_peered":"2024-04-24T23:11:02.196485+0000","last_clean":"2024-04-24T23:11:02.196485+0000","last_became_active":"2024-04-24T23:11:01.209482+0000","last_became_peered":"2024-04-24T23:11:01.209482+0000","last_unstale":"2024-04-24T23:11:02.196485+0000","last_undegraded":"2024-04-24T23:11:02.196485+0000","last_fullsized":"2024-04-24T23:11:02.196485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:01.407263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.199204+0000","last_change":"2024-04-24T23:11:00.305671+0000","last_active":"2024-04-24T23:11:01.199204+0000","last_peered":"2024-04-24T23:11:01.199204+0000","last_clean":"2024-04-24T23:11:01.199204+0000","last_became_active":"2024-04-24T23:11:00.303016+0000","last_became_peered":"2024-04-24T23:11:00.303016+0000","last_unstale":"2024-04-24T23:11:01.199204+0000","last_undegraded":"2024-04-24T23:11:01.199204+0000","last_fullsized":"2024-04-24T23:11:01.199204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:33.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618791+0000","last_change":"2024-04-24T23:11:01.206269+0000","last_active":"2024-04-24T23:11:02.618791+0000","last_peered":"2024-04-24T23:11:02.618791+0000","last_clean":"2024-04-24T23:11:02.618791+0000","last_became_active":"2024-04-24T23:11:01.205941+0000","last_became_peered":"2024-04-24T23:11:01.205941+0000","last_unstale":"2024-04-24T23:11:02.618791+0000","last_undegraded":"2024-04-24T23:11:02.618791+0000","last_fullsized":"2024-04-24T23:11:02.618791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:28.955247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:03.334458+0000","last_change":"2024-04-24T23:11:00.306071+0000","last_active":"2024-04-24T23:11:03.334458+0000","last_peered":"2024-04-24T23:11:03.334458+0000","last_clean":"2024-04-24T23:11:03.334458+0000","last_became_active":"2024-04-24T23:11:00.305751+0000","last_became_peered":"2024-04-24T23:11:00.305751+0000","last_unstale":"2024-04-24T23:11:03.334458+0000","last_undegraded":"2024-04-24T23:11:03.334458+0000","last_fullsized":"2024-04-24T23:11:03.334458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:30.675323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.222931+0000","last_change":"2024-04-24T23:11:01.222118+0000","last_active":"2024-04-24T23:11:01.222931+0000","last_peered":"2024-04-24T23:11:01.222931+0000","last_clean":"2024-04-24T23:11:01.222931+0000","last_became_active":"2024-04-24T23:11:01.221830+0000","last_became_peered":"2024-04-24T23:11:01.221830+0000","last_unstale":"2024-04-24T23:11:01.222931+0000","last_undegraded":"2024-04-24T23:11:01.222931+0000","last_fullsized":"2024-04-24T23:11:01.222931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:50.538146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618140+0000","last_change":"2024-04-24T23:11:00.308662+0000","last_active":"2024-04-24T23:11:02.618140+0000","last_peered":"2024-04-24T23:11:02.618140+0000","last_clean":"2024-04-24T23:11:02.618140+0000","last_became_active":"2024-04-24T23:11:00.308355+0000","last_became_peered":"2024-04-24T23:11:00.308355+0000","last_unstale":"2024-04-24T23:11:02.618140+0000","last_undegraded":"2024-04-24T23:11:02.618140+0000","last_fullsized":"2024-04-24T23:11:02.618140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:15.228277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618155+0000","last_change":"2024-04-24T23:11:01.210740+0000","last_active":"2024-04-24T23:11:02.618155+0000","last_peered":"2024-04-24T23:11:02.618155+0000","last_clean":"2024-04-24T23:11:02.618155+0000","last_became_active":"2024-04-24T23:11:01.210188+0000","last_became_peered":"2024-04-24T23:11:01.210188+0000","last_unstale":"2024-04-24T23:11:02.618155+0000","last_undegraded":"2024-04-24T23:11:02.618155+0000","last_fullsized":"2024-04-24T23:11:02.618155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:17.821343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.303043+0000","last_change":"2024-04-24T23:11:00.308114+0000","last_active":"2024-04-24T23:11:01.303043+0000","last_peered":"2024-04-24T23:11:01.303043+0000","last_clean":"2024-04-24T23:11:01.303043+0000","last_became_active":"2024-04-24T23:11:00.306632+0000","last_became_peered":"2024-04-24T23:11:00.306632+0000","last_unstale":"2024-04-24T23:11:01.303043+0000","last_undegraded":"2024-04-24T23:11:01.303043+0000","last_fullsized":"2024-04-24T23:11:01.303043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:19.298736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.221274+0000","last_change":"2024-04-24T23:11:01.219789+0000","last_active":"2024-04-24T23:11:01.221274+0000","last_peered":"2024-04-24T23:11:01.221274+0000","last_clean":"2024-04-24T23:11:01.221274+0000","last_became_active":"2024-04-24T23:11:01.219498+0000","last_became_peered":"2024-04-24T23:11:01.219498+0000","last_unstale":"2024-04-24T23:11:01.221274+0000","last_undegraded":"2024-04-24T23:11:01.221274+0000","last_fullsized":"2024-04-24T23:11:01.221274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:49.970335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:05.203634+0000","last_change":"2024-04-24T23:11:00.308847+0000","last_active":"2024-04-24T23:11:05.203634+0000","last_peered":"2024-04-24T23:11:05.203634+0000","last_clean":"2024-04-24T23:11:05.203634+0000","last_became_active":"2024-04-24T23:11:00.308493+0000","last_became_peered":"2024-04-24T23:11:00.308493+0000","last_unstale":"2024-04-24T23:11:05.203634+0000","last_undegraded":"2024-04-24T23:11:05.203634+0000","last_fullsized":"2024-04-24T23:11:05.203634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:24.354679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618728+0000","last_change":"2024-04-24T23:11:01.204997+0000","last_active":"2024-04-24T23:11:02.618728+0000","last_peered":"2024-04-24T23:11:02.618728+0000","last_clean":"2024-04-24T23:11:02.618728+0000","last_became_active":"2024-04-24T23:11:01.204645+0000","last_became_peered":"2024-04-24T23:11:01.204645+0000","last_unstale":"2024-04-24T23:11:02.618728+0000","last_undegraded":"2024-04-24T23:11:02.618728+0000","last_fullsized":"2024-04-24T23:11:02.618728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:07.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.342798+0000","last_change":"2024-04-24T23:11:00.302465+0000","last_active":"2024-04-24T23:11:04.342798+0000","last_peered":"2024-04-24T23:11:04.342798+0000","last_clean":"2024-04-24T23:11:04.342798+0000","last_became_active":"2024-04-24T23:11:00.302162+0000","last_became_peered":"2024-04-24T23:11:00.302162+0000","last_unstale":"2024-04-24T23:11:04.342798+0000","last_undegraded":"2024-04-24T23:11:04.342798+0000","last_fullsized":"2024-04-24T23:11:04.342798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:51.352872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.209883+0000","last_change":"2024-04-24T23:11:01.206588+0000","last_active":"2024-04-24T23:11:01.209883+0000","last_peered":"2024-04-24T23:11:01.209883+0000","last_clean":"2024-04-24T23:11:01.209883+0000","last_became_active":"2024-04-24T23:11:01.206258+0000","last_became_peered":"2024-04-24T23:11:01.206258+0000","last_unstale":"2024-04-24T23:11:01.209883+0000","last_undegraded":"2024-04-24T23:11:01.209883+0000","last_fullsized":"2024-04-24T23:11:01.209883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:44.917402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.198145+0000","last_change":"2024-04-24T23:11:00.305179+0000","last_active":"2024-04-24T23:11:01.198145+0000","last_peered":"2024-04-24T23:11:01.198145+0000","last_clean":"2024-04-24T23:11:01.198145+0000","last_became_active":"2024-04-24T23:11:00.302951+0000","last_became_peered":"2024-04-24T23:11:00.302951+0000","last_unstale":"2024-04-24T23:11:01.198145+0000","last_undegraded":"2024-04-24T23:11:01.198145+0000","last_fullsized":"2024-04-24T23:11:01.198145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:33.278498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618416+0000","last_change":"2024-04-24T23:11:01.211062+0000","last_active":"2024-04-24T23:11:02.618416+0000","last_peered":"2024-04-24T23:11:02.618416+0000","last_clean":"2024-04-24T23:11:02.618416+0000","last_became_active":"2024-04-24T23:11:01.210752+0000","last_became_peered":"2024-04-24T23:11:01.210752+0000","last_unstale":"2024-04-24T23:11:02.618416+0000","last_undegraded":"2024-04-24T23:11:02.618416+0000","last_fullsized":"2024-04-24T23:11:02.618416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:54.790212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.199412+0000","last_change":"2024-04-24T23:11:00.305690+0000","last_active":"2024-04-24T23:11:01.199412+0000","last_peered":"2024-04-24T23:11:01.199412+0000","last_clean":"2024-04-24T23:11:01.199412+0000","last_became_active":"2024-04-24T23:11:00.305386+0000","last_became_peered":"2024-04-24T23:11:00.305386+0000","last_unstale":"2024-04-24T23:11:01.199412+0000","last_undegraded":"2024-04-24T23:11:01.199412+0000","last_fullsized":"2024-04-24T23:11:01.199412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:39.898085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617802+0000","last_change":"2024-04-24T23:11:01.208916+0000","last_active":"2024-04-24T23:11:02.617802+0000","last_peered":"2024-04-24T23:11:02.617802+0000","last_clean":"2024-04-24T23:11:02.617802+0000","last_became_active":"2024-04-24T23:11:01.208593+0000","last_became_peered":"2024-04-24T23:11:01.208593+0000","last_unstale":"2024-04-24T23:11:02.617802+0000","last_undegraded":"2024-04-24T23:11:02.617802+0000","last_fullsized":"2024-04-24T23:11:02.617802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:58.442365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618001+0000","last_change":"2024-04-24T23:11:00.309608+0000","last_active":"2024-04-24T23:11:02.618001+0000","last_peered":"2024-04-24T23:11:02.618001+0000","last_clean":"2024-04-24T23:11:02.618001+0000","last_became_active":"2024-04-24T23:11:00.309310+0000","last_became_peered":"2024-04-24T23:11:00.309310+0000","last_unstale":"2024-04-24T23:11:02.618001+0000","last_undegraded":"2024-04-24T23:11:02.618001+0000","last_fullsized":"2024-04-24T23:11:02.618001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:23.330558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.213567+0000","last_change":"2024-04-24T23:11:01.212376+0000","last_active":"2024-04-24T23:11:01.213567+0000","last_peered":"2024-04-24T23:11:01.213567+0000","last_clean":"2024-04-24T23:11:01.213567+0000","last_became_active":"2024-04-24T23:11:01.212043+0000","last_became_peered":"2024-04-24T23:11:01.212043+0000","last_unstale":"2024-04-24T23:11:01.213567+0000","last_undegraded":"2024-04-24T23:11:01.213567+0000","last_fullsized":"2024-04-24T23:11:01.213567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:16:22.086023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618696+0000","last_change":"2024-04-24T23:11:00.305483+0000","last_active":"2024-04-24T23:11:02.618696+0000","last_peered":"2024-04-24T23:11:02.618696+0000","last_clean":"2024-04-24T23:11:02.618696+0000","last_became_active":"2024-04-24T23:11:00.305186+0000","last_became_peered":"2024-04-24T23:11:00.305186+0000","last_unstale":"2024-04-24T23:11:02.618696+0000","last_undegraded":"2024-04-24T23:11:02.618696+0000","last_fullsized":"2024-04-24T23:11:02.618696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:55.633200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618628+0000","last_change":"2024-04-24T23:11:01.210803+0000","last_active":"2024-04-24T23:11:02.618628+0000","last_peered":"2024-04-24T23:11:02.618628+0000","last_clean":"2024-04-24T23:11:02.618628+0000","last_became_active":"2024-04-24T23:11:01.210154+0000","last_became_peered":"2024-04-24T23:11:01.210154+0000","last_unstale":"2024-04-24T23:11:02.618628+0000","last_undegraded":"2024-04-24T23:11:02.618628+0000","last_fullsized":"2024-04-24T23:11:02.618628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:08.106252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618294+0000","last_change":"2024-04-24T23:11:00.305518+0000","last_active":"2024-04-24T23:11:02.618294+0000","last_peered":"2024-04-24T23:11:02.618294+0000","last_clean":"2024-04-24T23:11:02.618294+0000","last_became_active":"2024-04-24T23:11:00.305192+0000","last_became_peered":"2024-04-24T23:11:00.305192+0000","last_unstale":"2024-04-24T23:11:02.618294+0000","last_undegraded":"2024-04-24T23:11:02.618294+0000","last_fullsized":"2024-04-24T23:11:02.618294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:43:36.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618846+0000","last_change":"2024-04-24T23:11:01.207723+0000","last_active":"2024-04-24T23:11:02.618846+0000","last_peered":"2024-04-24T23:11:02.618846+0000","last_clean":"2024-04-24T23:11:02.618846+0000","last_became_active":"2024-04-24T23:11:01.207431+0000","last_became_peered":"2024-04-24T23:11:01.207431+0000","last_unstale":"2024-04-24T23:11:02.618846+0000","last_undegraded":"2024-04-24T23:11:02.618846+0000","last_fullsized":"2024-04-24T23:11:02.618846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:06.883404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618740+0000","last_change":"2024-04-24T23:11:00.310700+0000","last_active":"2024-04-24T23:11:02.618740+0000","last_peered":"2024-04-24T23:11:02.618740+0000","last_clean":"2024-04-24T23:11:02.618740+0000","last_became_active":"2024-04-24T23:11:00.310216+0000","last_became_peered":"2024-04-24T23:11:00.310216+0000","last_unstale":"2024-04-24T23:11:02.618740+0000","last_undegraded":"2024-04-24T23:11:02.618740+0000","last_fullsized":"2024-04-24T23:11:02.618740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:25.326695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618679+0000","last_change":"2024-04-24T23:11:01.214385+0000","last_active":"2024-04-24T23:11:02.618679+0000","last_peered":"2024-04-24T23:11:02.618679+0000","last_clean":"2024-04-24T23:11:02.618679+0000","last_became_active":"2024-04-24T23:11:01.214090+0000","last_became_peered":"2024-04-24T23:11:01.214090+0000","last_unstale":"2024-04-24T23:11:02.618679+0000","last_undegraded":"2024-04-24T23:11:02.618679+0000","last_fullsized":"2024-04-24T23:11:02.618679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:57.024833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.211107+0000","last_change":"2024-04-24T23:11:01.209826+0000","last_active":"2024-04-24T23:11:01.211107+0000","last_peered":"2024-04-24T23:11:01.211107+0000","last_clean":"2024-04-24T23:11:01.211107+0000","last_became_active":"2024-04-24T23:11:01.209432+0000","last_became_peered":"2024-04-24T23:11:01.209432+0000","last_unstale":"2024-04-24T23:11:01.211107+0000","last_undegraded":"2024-04-24T23:11:01.211107+0000","last_fullsized":"2024-04-24T23:11:01.211107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:43.176698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.193237+0000","last_change":"2024-04-24T23:11:00.305349+0000","last_active":"2024-04-24T23:11:01.193237+0000","last_peered":"2024-04-24T23:11:01.193237+0000","last_clean":"2024-04-24T23:11:01.193237+0000","last_became_active":"2024-04-24T23:11:00.305046+0000","last_became_peered":"2024-04-24T23:11:00.305046+0000","last_unstale":"2024-04-24T23:11:01.193237+0000","last_undegraded":"2024-04-24T23:11:01.193237+0000","last_fullsized":"2024-04-24T23:11:01.193237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:56.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196882+0000","last_change":"2024-04-24T23:11:01.216209+0000","last_active":"2024-04-24T23:11:02.196882+0000","last_peered":"2024-04-24T23:11:02.196882+0000","last_clean":"2024-04-24T23:11:02.196882+0000","last_became_active":"2024-04-24T23:11:01.215899+0000","last_became_peered":"2024-04-24T23:11:01.215899+0000","last_unstale":"2024-04-24T23:11:02.196882+0000","last_undegraded":"2024-04-24T23:11:02.196882+0000","last_fullsized":"2024-04-24T23:11:02.196882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:09.404193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.199588+0000","last_change":"2024-04-24T23:11:00.303160+0000","last_active":"2024-04-24T23:11:01.199588+0000","last_peered":"2024-04-24T23:11:01.199588+0000","last_clean":"2024-04-24T23:11:01.199588+0000","last_became_active":"2024-04-24T23:11:00.302781+0000","last_became_peered":"2024-04-24T23:11:00.302781+0000","last_unstale":"2024-04-24T23:11:01.199588+0000","last_undegraded":"2024-04-24T23:11:01.199588+0000","last_fullsized":"2024-04-24T23:11:01.199588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:42.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617863+0000","last_change":"2024-04-24T23:09:43.738402+0000","last_active":"2024-04-24T23:11:02.617863+0000","last_peered":"2024-04-24T23:11:02.617863+0000","last_clean":"2024-04-24T23:11:02.617863+0000","last_became_active":"2024-04-24T23:09:43.737979+0000","last_became_peered":"2024-04-24T23:09:43.737979+0000","last_unstale":"2024-04-24T23:11:02.617863+0000","last_undegraded":"2024-04-24T23:11:02.617863+0000","last_fullsized":"2024-04-24T23:11:02.617863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:16.456818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:16.456818+0000","last_clean_scrub_stamp":"2024-04-24T23:06:16.456818+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:52.449055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.209041+0000","last_change":"2024-04-24T23:11:01.203542+0000","last_active":"2024-04-24T23:11:01.209041+0000","last_peered":"2024-04-24T23:11:01.209041+0000","last_clean":"2024-04-24T23:11:01.209041+0000","last_became_active":"2024-04-24T23:11:01.203111+0000","last_became_peered":"2024-04-24T23:11:01.203111+0000","last_unstale":"2024-04-24T23:11:01.209041+0000","last_undegraded":"2024-04-24T23:11:01.209041+0000","last_fullsized":"2024-04-24T23:11:01.209041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:36.020377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618100+0000","last_change":"2024-04-24T23:11:00.304542+0000","last_active":"2024-04-24T23:11:02.618100+0000","last_peered":"2024-04-24T23:11:02.618100+0000","last_clean":"2024-04-24T23:11:02.618100+0000","last_became_active":"2024-04-24T23:11:00.304234+0000","last_became_peered":"2024-04-24T23:11:00.304234+0000","last_unstale":"2024-04-24T23:11:02.618100+0000","last_undegraded":"2024-04-24T23:11:02.618100+0000","last_fullsized":"2024-04-24T23:11:02.618100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:18.318876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618696+0000","last_change":"2024-04-24T23:11:00.310962+0000","last_active":"2024-04-24T23:11:02.618696+0000","last_peered":"2024-04-24T23:11:02.618696+0000","last_clean":"2024-04-24T23:11:02.618696+0000","last_became_active":"2024-04-24T23:11:00.310648+0000","last_became_peered":"2024-04-24T23:11:00.310648+0000","last_unstale":"2024-04-24T23:11:02.618696+0000","last_undegraded":"2024-04-24T23:11:02.618696+0000","last_fullsized":"2024-04-24T23:11:02.618696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:01.693517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.207975+0000","last_change":"2024-04-24T23:11:01.200293+0000","last_active":"2024-04-24T23:11:01.207975+0000","last_peered":"2024-04-24T23:11:01.207975+0000","last_clean":"2024-04-24T23:11:01.207975+0000","last_became_active":"2024-04-24T23:11:01.199945+0000","last_became_peered":"2024-04-24T23:11:01.199945+0000","last_unstale":"2024-04-24T23:11:01.207975+0000","last_undegraded":"2024-04-24T23:11:01.207975+0000","last_fullsized":"2024-04-24T23:11:01.207975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:05.561591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.343471+0000","last_change":"2024-04-24T23:11:00.307467+0000","last_active":"2024-04-24T23:11:04.343471+0000","last_peered":"2024-04-24T23:11:04.343471+0000","last_clean":"2024-04-24T23:11:04.343471+0000","last_became_active":"2024-04-24T23:11:00.307147+0000","last_became_peered":"2024-04-24T23:11:00.307147+0000","last_unstale":"2024-04-24T23:11:04.343471+0000","last_undegraded":"2024-04-24T23:11:04.343471+0000","last_fullsized":"2024-04-24T23:11:04.343471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:42.955410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.197072+0000","last_change":"2024-04-24T23:11:01.210541+0000","last_active":"2024-04-24T23:11:02.197072+0000","last_peered":"2024-04-24T23:11:02.197072+0000","last_clean":"2024-04-24T23:11:02.197072+0000","last_became_active":"2024-04-24T23:11:01.210187+0000","last_became_peered":"2024-04-24T23:11:01.210187+0000","last_unstale":"2024-04-24T23:11:02.197072+0000","last_undegraded":"2024-04-24T23:11:02.197072+0000","last_fullsized":"2024-04-24T23:11:02.197072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:45.138001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.300387+0000","last_change":"2024-04-24T23:11:00.308782+0000","last_active":"2024-04-24T23:11:01.300387+0000","last_peered":"2024-04-24T23:11:01.300387+0000","last_clean":"2024-04-24T23:11:01.300387+0000","last_became_active":"2024-04-24T23:11:00.308269+0000","last_became_peered":"2024-04-24T23:11:00.308269+0000","last_unstale":"2024-04-24T23:11:01.300387+0000","last_undegraded":"2024-04-24T23:11:01.300387+0000","last_fullsized":"2024-04-24T23:11:01.300387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:05.089037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617990+0000","last_change":"2024-04-24T23:11:01.214723+0000","last_active":"2024-04-24T23:11:02.617990+0000","last_peered":"2024-04-24T23:11:02.617990+0000","last_clean":"2024-04-24T23:11:02.617990+0000","last_became_active":"2024-04-24T23:11:01.213570+0000","last_became_peered":"2024-04-24T23:11:01.213570+0000","last_unstale":"2024-04-24T23:11:02.617990+0000","last_undegraded":"2024-04-24T23:11:02.617990+0000","last_fullsized":"2024-04-24T23:11:02.617990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:52.258640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.208683+0000","last_change":"2024-04-24T23:11:00.307522+0000","last_active":"2024-04-24T23:11:01.208683+0000","last_peered":"2024-04-24T23:11:01.208683+0000","last_clean":"2024-04-24T23:11:01.208683+0000","last_became_active":"2024-04-24T23:11:00.306106+0000","last_became_peered":"2024-04-24T23:11:00.306106+0000","last_unstale":"2024-04-24T23:11:01.208683+0000","last_undegraded":"2024-04-24T23:11:01.208683+0000","last_fullsized":"2024-04-24T23:11:01.208683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:21.623996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618057+0000","last_change":"2024-04-24T23:11:01.207426+0000","last_active":"2024-04-24T23:11:02.618057+0000","last_peered":"2024-04-24T23:11:02.618057+0000","last_clean":"2024-04-24T23:11:02.618057+0000","last_became_active":"2024-04-24T23:11:01.207020+0000","last_became_peered":"2024-04-24T23:11:01.207020+0000","last_unstale":"2024-04-24T23:11:02.618057+0000","last_undegraded":"2024-04-24T23:11:02.618057+0000","last_fullsized":"2024-04-24T23:11:02.618057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:58.389998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:03.332255+0000","last_change":"2024-04-24T23:11:00.302686+0000","last_active":"2024-04-24T23:11:03.332255+0000","last_peered":"2024-04-24T23:11:03.332255+0000","last_clean":"2024-04-24T23:11:03.332255+0000","last_became_active":"2024-04-24T23:11:00.302362+0000","last_became_peered":"2024-04-24T23:11:00.302362+0000","last_unstale":"2024-04-24T23:11:03.332255+0000","last_undegraded":"2024-04-24T23:11:03.332255+0000","last_fullsized":"2024-04-24T23:11:03.332255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:30.402780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618174+0000","last_change":"2024-04-24T23:11:01.212413+0000","last_active":"2024-04-24T23:11:02.618174+0000","last_peered":"2024-04-24T23:11:02.618174+0000","last_clean":"2024-04-24T23:11:02.618174+0000","last_became_active":"2024-04-24T23:11:01.211975+0000","last_became_peered":"2024-04-24T23:11:01.211975+0000","last_unstale":"2024-04-24T23:11:02.618174+0000","last_undegraded":"2024-04-24T23:11:02.618174+0000","last_fullsized":"2024-04-24T23:11:02.618174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:56.129977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.315735+0000","last_change":"2024-04-24T23:11:00.306583+0000","last_active":"2024-04-24T23:11:02.315735+0000","last_peered":"2024-04-24T23:11:02.315735+0000","last_clean":"2024-04-24T23:11:02.315735+0000","last_became_active":"2024-04-24T23:11:00.306274+0000","last_became_peered":"2024-04-24T23:11:00.306274+0000","last_unstale":"2024-04-24T23:11:02.315735+0000","last_undegraded":"2024-04-24T23:11:02.315735+0000","last_fullsized":"2024-04-24T23:11:02.315735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:15:58.157736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196656+0000","last_change":"2024-04-24T23:11:01.214041+0000","last_active":"2024-04-24T23:11:02.196656+0000","last_peered":"2024-04-24T23:11:02.196656+0000","last_clean":"2024-04-24T23:11:02.196656+0000","last_became_active":"2024-04-24T23:11:01.213639+0000","last_became_peered":"2024-04-24T23:11:01.213639+0000","last_unstale":"2024-04-24T23:11:02.196656+0000","last_undegraded":"2024-04-24T23:11:02.196656+0000","last_fullsized":"2024-04-24T23:11:02.196656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:08.819111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618427+0000","last_change":"2024-04-24T23:11:00.303790+0000","last_active":"2024-04-24T23:11:02.618427+0000","last_peered":"2024-04-24T23:11:02.618427+0000","last_clean":"2024-04-24T23:11:02.618427+0000","last_became_active":"2024-04-24T23:11:00.303381+0000","last_became_peered":"2024-04-24T23:11:00.303381+0000","last_unstale":"2024-04-24T23:11:02.618427+0000","last_undegraded":"2024-04-24T23:11:02.618427+0000","last_fullsized":"2024-04-24T23:11:02.618427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:18.725608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618846+0000","last_change":"2024-04-24T23:11:01.219575+0000","last_active":"2024-04-24T23:11:02.618846+0000","last_peered":"2024-04-24T23:11:02.618846+0000","last_clean":"2024-04-24T23:11:02.618846+0000","last_became_active":"2024-04-24T23:11:01.218750+0000","last_became_peered":"2024-04-24T23:11:01.218750+0000","last_unstale":"2024-04-24T23:11:02.618846+0000","last_undegraded":"2024-04-24T23:11:02.618846+0000","last_fullsized":"2024-04-24T23:11:02.618846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:00.945493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:04.343007+0000","last_change":"2024-04-24T23:11:00.310468+0000","last_active":"2024-04-24T23:11:04.343007+0000","last_peered":"2024-04-24T23:11:04.343007+0000","last_clean":"2024-04-24T23:11:04.343007+0000","last_became_active":"2024-04-24T23:11:00.310176+0000","last_became_peered":"2024-04-24T23:11:00.310176+0000","last_unstale":"2024-04-24T23:11:04.343007+0000","last_undegraded":"2024-04-24T23:11:04.343007+0000","last_fullsized":"2024-04-24T23:11:04.343007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:25.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618709+0000","last_change":"2024-04-24T23:11:01.219593+0000","last_active":"2024-04-24T23:11:02.618709+0000","last_peered":"2024-04-24T23:11:02.618709+0000","last_clean":"2024-04-24T23:11:02.618709+0000","last_became_active":"2024-04-24T23:11:01.219247+0000","last_became_peered":"2024-04-24T23:11:01.219247+0000","last_unstale":"2024-04-24T23:11:02.618709+0000","last_undegraded":"2024-04-24T23:11:02.618709+0000","last_fullsized":"2024-04-24T23:11:02.618709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:12.605797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.300746+0000","last_change":"2024-04-24T23:11:00.305124+0000","last_active":"2024-04-24T23:11:01.300746+0000","last_peered":"2024-04-24T23:11:01.300746+0000","last_clean":"2024-04-24T23:11:01.300746+0000","last_became_active":"2024-04-24T23:11:00.301468+0000","last_became_peered":"2024-04-24T23:11:00.301468+0000","last_unstale":"2024-04-24T23:11:01.300746+0000","last_undegraded":"2024-04-24T23:11:01.300746+0000","last_fullsized":"2024-04-24T23:11:01.300746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:58.451956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618010+0000","last_change":"2024-04-24T23:11:01.209967+0000","last_active":"2024-04-24T23:11:02.618010+0000","last_peered":"2024-04-24T23:11:02.618010+0000","last_clean":"2024-04-24T23:11:02.618010+0000","last_became_active":"2024-04-24T23:11:01.209607+0000","last_became_peered":"2024-04-24T23:11:01.209607+0000","last_unstale":"2024-04-24T23:11:02.618010+0000","last_undegraded":"2024-04-24T23:11:02.618010+0000","last_fullsized":"2024-04-24T23:11:02.618010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:02.840211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.302788+0000","last_change":"2024-04-24T23:11:00.308200+0000","last_active":"2024-04-24T23:11:01.302788+0000","last_peered":"2024-04-24T23:11:01.302788+0000","last_clean":"2024-04-24T23:11:01.302788+0000","last_became_active":"2024-04-24T23:11:00.306369+0000","last_became_peered":"2024-04-24T23:11:00.306369+0000","last_unstale":"2024-04-24T23:11:01.302788+0000","last_undegraded":"2024-04-24T23:11:01.302788+0000","last_fullsized":"2024-04-24T23:11:01.302788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:24.462361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618700+0000","last_change":"2024-04-24T23:11:01.207706+0000","last_active":"2024-04-24T23:11:02.618700+0000","last_peered":"2024-04-24T23:11:02.618700+0000","last_clean":"2024-04-24T23:11:02.618700+0000","last_became_active":"2024-04-24T23:11:01.207357+0000","last_became_peered":"2024-04-24T23:11:01.207357+0000","last_unstale":"2024-04-24T23:11:02.618700+0000","last_undegraded":"2024-04-24T23:11:02.618700+0000","last_fullsized":"2024-04-24T23:11:02.618700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:46.785844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618382+0000","last_change":"2024-04-24T23:11:01.214886+0000","last_active":"2024-04-24T23:11:02.618382+0000","last_peered":"2024-04-24T23:11:02.618382+0000","last_clean":"2024-04-24T23:11:02.618382+0000","last_became_active":"2024-04-24T23:11:01.213671+0000","last_became_peered":"2024-04-24T23:11:01.213671+0000","last_unstale":"2024-04-24T23:11:02.618382+0000","last_undegraded":"2024-04-24T23:11:02.618382+0000","last_fullsized":"2024-04-24T23:11:02.618382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:44.777437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.316051+0000","last_change":"2024-04-24T23:11:01.203578+0000","last_active":"2024-04-24T23:11:02.316051+0000","last_peered":"2024-04-24T23:11:02.316051+0000","last_clean":"2024-04-24T23:11:02.316051+0000","last_became_active":"2024-04-24T23:11:01.203250+0000","last_became_peered":"2024-04-24T23:11:01.203250+0000","last_unstale":"2024-04-24T23:11:02.316051+0000","last_undegraded":"2024-04-24T23:11:02.316051+0000","last_fullsized":"2024-04-24T23:11:02.316051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:39.050149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.316102+0000","last_change":"2024-04-24T23:11:01.209641+0000","last_active":"2024-04-24T23:11:02.316102+0000","last_peered":"2024-04-24T23:11:02.316102+0000","last_clean":"2024-04-24T23:11:02.316102+0000","last_became_active":"2024-04-24T23:11:01.209110+0000","last_became_peered":"2024-04-24T23:11:01.209110+0000","last_unstale":"2024-04-24T23:11:02.316102+0000","last_undegraded":"2024-04-24T23:11:02.316102+0000","last_fullsized":"2024-04-24T23:11:02.316102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:14.709926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.617842+0000","last_change":"2024-04-24T23:11:01.213894+0000","last_active":"2024-04-24T23:11:02.617842+0000","last_peered":"2024-04-24T23:11:02.617842+0000","last_clean":"2024-04-24T23:11:02.617842+0000","last_became_active":"2024-04-24T23:11:01.213461+0000","last_became_peered":"2024-04-24T23:11:01.213461+0000","last_unstale":"2024-04-24T23:11:02.617842+0000","last_undegraded":"2024-04-24T23:11:02.617842+0000","last_fullsized":"2024-04-24T23:11:02.617842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:00.786413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.209228+0000","last_change":"2024-04-24T23:11:01.206556+0000","last_active":"2024-04-24T23:11:01.209228+0000","last_peered":"2024-04-24T23:11:01.209228+0000","last_clean":"2024-04-24T23:11:01.209228+0000","last_became_active":"2024-04-24T23:11:01.206178+0000","last_became_peered":"2024-04-24T23:11:01.206178+0000","last_unstale":"2024-04-24T23:11:01.209228+0000","last_undegraded":"2024-04-24T23:11:01.209228+0000","last_fullsized":"2024-04-24T23:11:01.209228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:00.175173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618585+0000","last_change":"2024-04-24T23:11:01.213174+0000","last_active":"2024-04-24T23:11:02.618585+0000","last_peered":"2024-04-24T23:11:02.618585+0000","last_clean":"2024-04-24T23:11:02.618585+0000","last_became_active":"2024-04-24T23:11:01.212815+0000","last_became_peered":"2024-04-24T23:11:01.212815+0000","last_unstale":"2024-04-24T23:11:02.618585+0000","last_undegraded":"2024-04-24T23:11:02.618585+0000","last_fullsized":"2024-04-24T23:11:02.618585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:09.281324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.316312+0000","last_change":"2024-04-24T23:11:01.215030+0000","last_active":"2024-04-24T23:11:02.316312+0000","last_peered":"2024-04-24T23:11:02.316312+0000","last_clean":"2024-04-24T23:11:02.316312+0000","last_became_active":"2024-04-24T23:11:01.214386+0000","last_became_peered":"2024-04-24T23:11:01.214386+0000","last_unstale":"2024-04-24T23:11:02.316312+0000","last_undegraded":"2024-04-24T23:11:02.316312+0000","last_fullsized":"2024-04-24T23:11:02.316312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:10.633565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196642+0000","last_change":"2024-04-24T23:11:01.217748+0000","last_active":"2024-04-24T23:11:02.196642+0000","last_peered":"2024-04-24T23:11:02.196642+0000","last_clean":"2024-04-24T23:11:02.196642+0000","last_became_active":"2024-04-24T23:11:01.217403+0000","last_became_peered":"2024-04-24T23:11:01.217403+0000","last_unstale":"2024-04-24T23:11:02.196642+0000","last_undegraded":"2024-04-24T23:11:02.196642+0000","last_fullsized":"2024-04-24T23:11:02.196642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:29.569179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.210197+0000","last_change":"2024-04-24T23:11:01.208894+0000","last_active":"2024-04-24T23:11:01.210197+0000","last_peered":"2024-04-24T23:11:01.210197+0000","last_clean":"2024-04-24T23:11:01.210197+0000","last_became_active":"2024-04-24T23:11:01.208576+0000","last_became_peered":"2024-04-24T23:11:01.208576+0000","last_unstale":"2024-04-24T23:11:01.210197+0000","last_undegraded":"2024-04-24T23:11:01.210197+0000","last_fullsized":"2024-04-24T23:11:01.210197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:55.852486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.196521+0000","last_change":"2024-04-24T23:11:01.218561+0000","last_active":"2024-04-24T23:11:02.196521+0000","last_peered":"2024-04-24T23:11:02.196521+0000","last_clean":"2024-04-24T23:11:02.196521+0000","last_became_active":"2024-04-24T23:11:01.218261+0000","last_became_peered":"2024-04-24T23:11:01.218261+0000","last_unstale":"2024-04-24T23:11:02.196521+0000","last_undegraded":"2024-04-24T23:11:02.196521+0000","last_fullsized":"2024-04-24T23:11:02.196521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:49.677225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.216200+0000","last_change":"2024-04-24T23:11:01.214868+0000","last_active":"2024-04-24T23:11:01.216200+0000","last_peered":"2024-04-24T23:11:01.216200+0000","last_clean":"2024-04-24T23:11:01.216200+0000","last_became_active":"2024-04-24T23:11:01.214582+0000","last_became_peered":"2024-04-24T23:11:01.214582+0000","last_unstale":"2024-04-24T23:11:01.216200+0000","last_undegraded":"2024-04-24T23:11:01.216200+0000","last_fullsized":"2024-04-24T23:11:01.216200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:38.201838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-24T23:11:01.215938+0000","last_change":"2024-04-24T23:11:01.212348+0000","last_active":"2024-04-24T23:11:01.215938+0000","last_peered":"2024-04-24T23:11:01.215938+0000","last_clean":"2024-04-24T23:11:01.215938+0000","last_became_active":"2024-04-24T23:11:01.211802+0000","last_became_peered":"2024-04-24T23:11:01.211802+0000","last_unstale":"2024-04-24T23:11:01.215938+0000","last_undegraded":"2024-04-24T23:11:01.215938+0000","last_fullsized":"2024-04-24T23:11:01.215938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:12.552821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T23:11:02.618276+0000","last_change":"2024-04-24T23:11:01.209834+0000","last_active":"2024-04-24T23:11:02.618276+0000","last_peered":"2024-04-24T23:11:02.618276+0000","last_clean":"2024-04-24T23:11:02.618276+0000","last_became_active":"2024-04-24T23:11:01.209476+0000","last_became_peered":"2024-04-24T23:11:01.209476+0000","last_unstale":"2024-04-24T23:11:02.618276+0000","last_undegraded":"2024-04-24T23:11:02.618276+0000","last_fullsized":"2024-04-24T23:11:02.618276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:20.036684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244416,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9772,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":81,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":24328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":913408,"data_stored":434445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1043564,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":443833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":435049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":1033965,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":434445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":425368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":434535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":445977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":443299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":425368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":1024813,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:11:07.886 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:11:07.887 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T23:11:07.887 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T23:11:08.156 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:08 smithi102 ceph-mon[29216]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:08.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:08 smithi161 ceph-mon[32709]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:08.616 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:08 smithi183 ceph-mon[33137]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:09 smithi102 ceph-mon[29216]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:09 smithi102 ceph-mon[29216]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:09 smithi102 ceph-mon[29216]: pgmap v296: 97 pgs: 97 active+clean; 589 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-04-24T23:11:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:09 smithi161 ceph-mon[32709]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:09.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:09 smithi161 ceph-mon[32709]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:09.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:09 smithi161 ceph-mon[32709]: pgmap v296: 97 pgs: 97 active+clean; 589 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-04-24T23:11:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:09 smithi183 ceph-mon[33137]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:09 smithi183 ceph-mon[33137]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:11:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:09 smithi183 ceph-mon[33137]: pgmap v296: 97 pgs: 97 active+clean; 589 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.819+0000 7ff26e545700 1 -- 172.21.15.102:0/3449996544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff268102d50 msgr2=0x7ff268109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.819+0000 7ff26e545700 1 --2- 172.21.15.102:0/3449996544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff268102d50 0x7ff268109600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff26000b0d0 tx=0x7ff26000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.819+0000 7ff26e545700 1 -- 172.21.15.102:0/3449996544 shutdown_connections 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.819+0000 7ff26e545700 1 --2- 172.21.15.102:0/3449996544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff268102d50 0x7ff268109600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.819+0000 7ff26e545700 1 --2- 172.21.15.102:0/3449996544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2681011b0 0x7ff2681015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.819+0000 7ff26e545700 1 --2- 172.21.15.102:0/3449996544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2681023b0 0x7ff268102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.819+0000 7ff26e545700 1 -- 172.21.15.102:0/3449996544 >> 172.21.15.102:0/3449996544 conn(0x7ff2680fc750 msgr2=0x7ff2680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.819+0000 7ff26e545700 1 -- 172.21.15.102:0/3449996544 shutdown_connections 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.819+0000 7ff26e545700 1 -- 172.21.15.102:0/3449996544 wait complete. 2024-04-24T23:11:09.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26e545700 1 Processor -- start 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26e545700 1 -- start start 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26e545700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2681011b0 0x7ff2681982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26e545700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2681023b0 0x7ff268198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26e545700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff268102d50 0x7ff26819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26e545700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff26810ccb0 con 0x7ff2681011b0 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26e545700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff26810cb30 con 0x7ff2681023b0 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26e545700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff26810ce30 con 0x7ff268102d50 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff267fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2681011b0 0x7ff2681982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff267fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2681011b0 0x7ff2681982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55726/0 (socket says 172.21.15.102:55726) 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff267fff700 1 -- 172.21.15.102:0/2005868191 learned_addr learned my addr 172.21.15.102:0/2005868191 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff25ffff700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2681023b0 0x7ff268198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26cae2700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff268102d50 0x7ff26819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff267fff700 1 -- 172.21.15.102:0/2005868191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff268102d50 msgr2=0x7ff26819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff267fff700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff268102d50 0x7ff26819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff267fff700 1 -- 172.21.15.102:0/2005868191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2681023b0 msgr2=0x7ff268198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff267fff700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2681023b0 0x7ff268198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff267fff700 1 -- 172.21.15.102:0/2005868191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26819ff40 con 0x7ff2681011b0 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff26cae2700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff268102d50 0x7ff26819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.820+0000 7ff267fff700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2681011b0 0x7ff2681982e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c9a0 tx=0x7ff25800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.821+0000 7ff265ffb700 1 -- 172.21.15.102:0/2005868191 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258004220 con 0x7ff2681011b0 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.821+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2681a0230 con 0x7ff2681011b0 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.821+0000 7ff265ffb700 1 -- 172.21.15.102:0/2005868191 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff258007d10 con 0x7ff2681011b0 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.821+0000 7ff265ffb700 1 -- 172.21.15.102:0/2005868191 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258003d00 con 0x7ff2681011b0 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.821+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2681a07f0 con 0x7ff2681011b0 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.822+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2680507a0 con 0x7ff2681011b0 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.823+0000 7ff265ffb700 1 -- 172.21.15.102:0/2005868191 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff258007860 con 0x7ff2681011b0 2024-04-24T23:11:09.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.823+0000 7ff265ffb700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff2540a2d60 0x7ff2540a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:09.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.823+0000 7ff25ffff700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff2540a2d60 0x7ff2540a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:09.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.823+0000 7ff265ffb700 1 -- 172.21.15.102:0/2005868191 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff2580cf020 con 0x7ff2681011b0 2024-04-24T23:11:09.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.824+0000 7ff25ffff700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff2540a2d60 0x7ff2540a5220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff250009a10 tx=0x7ff250009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:09.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:09.827+0000 7ff265ffb700 1 -- 172.21.15.102:0/2005868191 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff258014030 con 0x7ff2681011b0 2024-04-24T23:11:10.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.108+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7ff2681a0cf0 con 0x7ff2681011b0 2024-04-24T23:11:10.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.110+0000 7ff265ffb700 1 -- 172.21.15.102:0/2005868191 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7ff25809bfd0 con 0x7ff2681011b0 2024-04-24T23:11:10.112 INFO:teuthology.orchestra.run.smithi102.stdout:[client.0] 2024-04-24T23:11:10.112 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQCOkSlmucaMBhAAD7nffTKgz6Fos1XuGzEvug== 2024-04-24T23:11:10.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff2540a2d60 msgr2=0x7ff2540a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff2540a2d60 0x7ff2540a5220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff250009a10 tx=0x7ff250009190 comp rx=0 tx=0).stop 2024-04-24T23:11:10.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2681011b0 msgr2=0x7ff2681982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2681011b0 0x7ff2681982e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c9a0 tx=0x7ff25800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 shutdown_connections 2024-04-24T23:11:10.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff268102d50 0x7ff26819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff2540a2d60 0x7ff2540a5220 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2681011b0 0x7ff2681982e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 --2- 172.21.15.102:0/2005868191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2681023b0 0x7ff268198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 >> 172.21.15.102:0/2005868191 conn(0x7ff2680fc750 msgr2=0x7ff2680fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:10.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 shutdown_connections 2024-04-24T23:11:10.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:10.112+0000 7ff26e545700 1 -- 172.21.15.102:0/2005868191 wait complete. 2024-04-24T23:11:10.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:10 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2005868191' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:11:10.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:10 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2005868191' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:11:10.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:10 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2005868191' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:11:10.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:10 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2005868191' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:11:10.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:10 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2005868191' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:11:10.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:10 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2005868191' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:11:10.938 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:11:10.938 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T23:11:10.938 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T23:11:10.979 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T23:11:10.979 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T23:11:10.979 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T23:11:11.258 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:11.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:11 smithi161 ceph-mon[32709]: pgmap v297: 97 pgs: 97 active+clean; 589 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-04-24T23:11:11.644 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:11 smithi102 ceph-mon[29216]: pgmap v297: 97 pgs: 97 active+clean; 589 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-04-24T23:11:11.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:11 smithi183 ceph-mon[33137]: pgmap v297: 97 pgs: 97 active+clean; 589 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 -- 172.21.15.102:0/3535125868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c106160 msgr2=0x7f0b7c106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 --2- 172.21.15.102:0/3535125868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c106160 0x7f0b7c106540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009990 tx=0x7f0b78009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 -- 172.21.15.102:0/3535125868 shutdown_connections 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 --2- 172.21.15.102:0/3535125868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b7c102250 0x7f0b7c10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 --2- 172.21.15.102:0/3535125868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c106160 0x7f0b7c106540 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 --2- 172.21.15.102:0/3535125868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7c101890 0x7f0b7c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 -- 172.21.15.102:0/3535125868 >> 172.21.15.102:0/3535125868 conn(0x7f0b7c077730 msgr2=0x7f0b7c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 -- 172.21.15.102:0/3535125868 shutdown_connections 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 -- 172.21.15.102:0/3535125868 wait complete. 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 Processor -- start 2024-04-24T23:11:13.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.052+0000 7f0b8388b700 1 -- start start 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b8388b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7c101890 0x7f0b7c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b8388b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c102250 0x7f0b7c198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b8388b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b7c106160 0x7f0b7c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b8388b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1112e0 con 0x7f0b7c102250 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b8388b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c111160 con 0x7f0b7c101890 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b8388b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c111460 con 0x7f0b7c106160 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b80e26700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c102250 0x7f0b7c198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b80e26700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c102250 0x7f0b7c198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55742/0 (socket says 172.21.15.102:55742) 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b80e26700 1 -- 172.21.15.102:0/431427862 learned_addr learned my addr 172.21.15.102:0/431427862 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b81e28700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b7c106160 0x7f0b7c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b81627700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7c101890 0x7f0b7c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b80e26700 1 -- 172.21.15.102:0/431427862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b7c106160 msgr2=0x7f0b7c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b80e26700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b7c106160 0x7f0b7c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b80e26700 1 -- 172.21.15.102:0/431427862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7c101890 msgr2=0x7f0b7c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b80e26700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7c101890 0x7f0b7c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b80e26700 1 -- 172.21.15.102:0/431427862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c19d740 con 0x7f0b7c102250 2024-04-24T23:11:13.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b80e26700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c102250 0x7f0b7c198cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c9b0 tx=0x7f0b6c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b727fc700 1 -- 172.21.15.102:0/431427862 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c007b00 con 0x7f0b7c102250 2024-04-24T23:11:13.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b727fc700 1 -- 172.21.15.102:0/431427862 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b6c007ca0 con 0x7f0b7c102250 2024-04-24T23:11:13.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c19da30 con 0x7f0b7c102250 2024-04-24T23:11:13.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b727fc700 1 -- 172.21.15.102:0/431427862 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c018660 con 0x7f0b7c102250 2024-04-24T23:11:13.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.053+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c1a52b0 con 0x7f0b7c102250 2024-04-24T23:11:13.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.054+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c068f20 con 0x7f0b7c102250 2024-04-24T23:11:13.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.056+0000 7f0b727fc700 1 -- 172.21.15.102:0/431427862 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0b6c00f450 con 0x7f0b7c102250 2024-04-24T23:11:13.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.057+0000 7f0b727fc700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0b680a2cb0 0x7f0b680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:13.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.057+0000 7f0b81627700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0b680a2cb0 0x7f0b680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:13.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.057+0000 7f0b727fc700 1 -- 172.21.15.102:0/431427862 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0b6c0cf260 con 0x7f0b7c102250 2024-04-24T23:11:13.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.057+0000 7f0b81627700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0b680a2cb0 0x7f0b680a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0b78006010 tx=0x7f0b7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:13.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.061+0000 7f0b727fc700 1 -- 172.21.15.102:0/431427862 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0b6c014030 con 0x7f0b7c102250 2024-04-24T23:11:13.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.313+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0b7c02db30 con 0x7f0b7c102250 2024-04-24T23:11:13.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.317+0000 7f0b727fc700 1 -- 172.21.15.102:0/431427862 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263209 (secure 0 0 0) 0x7f0b6c09c2a0 con 0x7f0b7c102250 2024-04-24T23:11:13.323 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:11:13.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0b680a2cb0 msgr2=0x7f0b680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0b680a2cb0 0x7f0b680a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0b78006010 tx=0x7f0b7801a040 comp rx=0 tx=0).stop 2024-04-24T23:11:13.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c102250 msgr2=0x7f0b7c198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:13.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c102250 0x7f0b7c198cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c9b0 tx=0x7f0b6c00ccc0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 shutdown_connections 2024-04-24T23:11:13.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b7c106160 0x7f0b7c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7c101890 0x7f0b7c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0b680a2cb0 0x7f0b680a5170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 --2- 172.21.15.102:0/431427862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b7c102250 0x7f0b7c198cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:13.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 >> 172.21.15.102:0/431427862 conn(0x7f0b7c077730 msgr2=0x7f0b7c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:13.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 shutdown_connections 2024-04-24T23:11:13.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:13.320+0000 7f0b8388b700 1 -- 172.21.15.102:0/431427862 wait complete. 2024-04-24T23:11:13.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:13 smithi102 ceph-mon[29216]: pgmap v298: 97 pgs: 97 active+clean; 618 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-04-24T23:11:13.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:13 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/431427862' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:11:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:13 smithi161 ceph-mon[32709]: pgmap v298: 97 pgs: 97 active+clean; 618 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-04-24T23:11:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:13 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/431427862' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:11:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:13 smithi183 ceph-mon[33137]: pgmap v298: 97 pgs: 97 active+clean; 618 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-04-24T23:11:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:13 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/431427862' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:11:13.970 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3611055373},{"type":"v1","addr":"172.21.15.102:6801","nonce":3611055373}]},"active_addr":"172.21.15.102:6801/3611055373","active_change":"2024-04-24T23:03:48.134831+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.102:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":1812533171}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":2090298312}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":835175442}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":4149172465}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":4021738971}]}]} 2024-04-24T23:11:13.977 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T23:11:13.977 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T23:11:13.978 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:11:14.218 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:15.606 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:15 smithi102 ceph-mon[29216]: pgmap v299: 97 pgs: 97 active+clean; 618 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-04-24T23:11:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:15 smithi161 ceph-mon[32709]: pgmap v299: 97 pgs: 97 active+clean; 618 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-04-24T23:11:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:15 smithi183 ceph-mon[33137]: pgmap v299: 97 pgs: 97 active+clean; 618 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-04-24T23:11:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.003+0000 7fc76f81e700 1 -- 172.21.15.102:0/1142424293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768107190 msgr2=0x7fc768107570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.003+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1142424293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768107190 0x7fc768107570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc764009990 tx=0x7fc764009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.003+0000 7fc76f81e700 1 -- 172.21.15.102:0/1142424293 shutdown_connections 2024-04-24T23:11:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.003+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1142424293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc768101ae0 0x7fc76810dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.003+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1142424293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768107190 0x7fc768107570 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.003+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1142424293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc768101120 0x7fc7681015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.003+0000 7fc76f81e700 1 -- 172.21.15.102:0/1142424293 >> 172.21.15.102:0/1142424293 conn(0x7fc7680780a0 msgr2=0x7fc7680784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.003+0000 7fc76f81e700 1 -- 172.21.15.102:0/1142424293 shutdown_connections 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.003+0000 7fc76f81e700 1 -- 172.21.15.102:0/1142424293 wait complete. 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76f81e700 1 Processor -- start 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76f81e700 1 -- start start 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76f81e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc768101120 0x7fc76819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76f81e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768101ae0 0x7fc76819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76f81e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc768107190 0x7fc768197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76f81e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681113b0 con 0x7fc768101ae0 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76f81e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc768111230 con 0x7fc768107190 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76f81e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc768111530 con 0x7fc768101120 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76cdb9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768101ae0 0x7fc76819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76cdb9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768101ae0 0x7fc76819dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55764/0 (socket says 172.21.15.102:55764) 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76cdb9700 1 -- 172.21.15.102:0/1040428775 learned_addr learned my addr 172.21.15.102:0/1040428775 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76d5ba700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc768101120 0x7fc76819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76cdb9700 1 -- 172.21.15.102:0/1040428775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc768101120 msgr2=0x7fc76819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76cdb9700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc768101120 0x7fc76819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76cdb9700 1 -- 172.21.15.102:0/1040428775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc768107190 msgr2=0x7fc768197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76ddbb700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc768107190 0x7fc768197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76cdb9700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc768107190 0x7fc768197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76cdb9700 1 -- 172.21.15.102:0/1040428775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc768198050 con 0x7fc768101ae0 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76ddbb700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc768107190 0x7fc768197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.004+0000 7fc76cdb9700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768101ae0 0x7fc76819dbe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc75800b530 tx=0x7fc75800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.005+0000 7fc75e7fc700 1 -- 172.21.15.102:0/1040428775 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758015020 con 0x7fc768101ae0 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.005+0000 7fc75e7fc700 1 -- 172.21.15.102:0/1040428775 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc758011ce0 con 0x7fc768101ae0 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.005+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc768198340 con 0x7fc768101ae0 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.005+0000 7fc75e7fc700 1 -- 172.21.15.102:0/1040428775 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758010d60 con 0x7fc768101ae0 2024-04-24T23:11:16.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.005+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7681a4620 con 0x7fc768101ae0 2024-04-24T23:11:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.006+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc768102840 con 0x7fc768101ae0 2024-04-24T23:11:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.007+0000 7fc75e7fc700 1 -- 172.21.15.102:0/1040428775 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc758018020 con 0x7fc768101ae0 2024-04-24T23:11:16.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.007+0000 7fc75e7fc700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7540a2cb0 0x7fc7540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:16.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.007+0000 7fc76d5ba700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7540a2cb0 0x7fc7540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:16.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.007+0000 7fc75e7fc700 1 -- 172.21.15.102:0/1040428775 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc7580ced20 con 0x7fc768101ae0 2024-04-24T23:11:16.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.008+0000 7fc76d5ba700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7540a2cb0 0x7fc7540a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc764006010 tx=0x7fc76401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:16.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.011+0000 7fc75e7fc700 1 -- 172.21.15.102:0/1040428775 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc75801f020 con 0x7fc768101ae0 2024-04-24T23:11:16.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.209+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc768198ca0 con 0x7fc768101ae0 2024-04-24T23:11:16.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.211+0000 7fc75e7fc700 1 -- 172.21.15.102:0/1040428775 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22279 (secure 0 0 0) 0x7fc75809bde0 con 0x7fc768101ae0 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":86,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:11:02.189206+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7540a2cb0 msgr2=0x7fc7540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7540a2cb0 0x7fc7540a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc764006010 tx=0x7fc76401a040 comp rx=0 tx=0).stop 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768101ae0 msgr2=0x7fc76819dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768101ae0 0x7fc76819dbe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc75800b530 tx=0x7fc75800b8f0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 shutdown_connections 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc768107190 0x7fc768197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc768101120 0x7fc76819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc7540a2cb0 0x7fc7540a5170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 --2- 172.21.15.102:0/1040428775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc768101ae0 0x7fc76819dbe0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:16.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 >> 172.21.15.102:0/1040428775 conn(0x7fc7680780a0 msgr2=0x7fc7680fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:16.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 shutdown_connections 2024-04-24T23:11:16.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:16.214+0000 7fc76f81e700 1 -- 172.21.15.102:0/1040428775 wait complete. 2024-04-24T23:11:16.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:16 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1040428775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:16.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:16 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1040428775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:16.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:16 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1040428775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:16.905 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T23:11:16.905 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:11:17.143 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:17.521 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:17 smithi102 ceph-mon[29216]: pgmap v300: 97 pgs: 97 active+clean; 625 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-04-24T23:11:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:17 smithi183 ceph-mon[33137]: pgmap v300: 97 pgs: 97 active+clean; 625 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-04-24T23:11:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:17 smithi161 ceph-mon[32709]: pgmap v300: 97 pgs: 97 active+clean; 625 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-04-24T23:11:18.562 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:18 smithi102 ceph-mon[29216]: pgmap v301: 97 pgs: 97 active+clean; 625 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-04-24T23:11:18.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:18 smithi183 ceph-mon[33137]: pgmap v301: 97 pgs: 97 active+clean; 625 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-04-24T23:11:18.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:18 smithi161 ceph-mon[32709]: pgmap v301: 97 pgs: 97 active+clean; 625 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2024-04-24T23:11:18.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.917+0000 7f124ae86700 1 -- 172.21.15.102:0/2882651206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 msgr2=0x7f124406a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.917+0000 7f124ae86700 1 --2- 172.21.15.102:0/2882651206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 0x7f124406a3a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1230009990 tx=0x7f1230009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.917+0000 7f124ae86700 1 -- 172.21.15.102:0/2882651206 shutdown_connections 2024-04-24T23:11:18.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.917+0000 7f124ae86700 1 --2- 172.21.15.102:0/2882651206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124406a8e0 0x7f124410f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.917+0000 7f124ae86700 1 --2- 172.21.15.102:0/2882651206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12441082b0 0x7f1244108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.917+0000 7f124ae86700 1 --2- 172.21.15.102:0/2882651206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 0x7f124406a3a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.917+0000 7f124ae86700 1 -- 172.21.15.102:0/2882651206 >> 172.21.15.102:0/2882651206 conn(0x7f12440780e0 msgr2=0x7f12440784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.917+0000 7f124ae86700 1 -- 172.21.15.102:0/2882651206 shutdown_connections 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.917+0000 7f124ae86700 1 -- 172.21.15.102:0/2882651206 wait complete. 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f124ae86700 1 Processor -- start 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f124ae86700 1 -- start start 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f124ae86700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 0x7f1244101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f124ae86700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124406a8e0 0x7f1244101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f124ae86700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12441082b0 0x7f1244101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f124ae86700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12441123c0 con 0x7f1244069f40 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f124ae86700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1244112240 con 0x7f12441082b0 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f124ae86700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1244112540 con 0x7f124406a8e0 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f1248c22700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 0x7f1244101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f1248c22700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 0x7f1244101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50374/0 (socket says 172.21.15.102:50374) 2024-04-24T23:11:18.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f1248c22700 1 -- 172.21.15.102:0/468725320 learned_addr learned my addr 172.21.15.102:0/468725320 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f1243fff700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124406a8e0 0x7f1244101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f1249423700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12441082b0 0x7f1244101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f1248c22700 1 -- 172.21.15.102:0/468725320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124406a8e0 msgr2=0x7f1244101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f1248c22700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124406a8e0 0x7f1244101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f1248c22700 1 -- 172.21.15.102:0/468725320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12441082b0 msgr2=0x7f1244101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.918+0000 7f1248c22700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12441082b0 0x7f1244101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.919+0000 7f1248c22700 1 -- 172.21.15.102:0/468725320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12441a6ea0 con 0x7f1244069f40 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.919+0000 7f1248c22700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 0x7f1244101200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f123800d980 tx=0x7f123800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.919+0000 7f1241ffb700 1 -- 172.21.15.102:0/468725320 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238017070 con 0x7f1244069f40 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.919+0000 7f1241ffb700 1 -- 172.21.15.102:0/468725320 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1238005020 con 0x7f1244069f40 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.919+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12441a7190 con 0x7f1244069f40 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.919+0000 7f1241ffb700 1 -- 172.21.15.102:0/468725320 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238009b50 con 0x7f1244069f40 2024-04-24T23:11:18.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.919+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12441a76a0 con 0x7f1244069f40 2024-04-24T23:11:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.920+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1244068f20 con 0x7f1244069f40 2024-04-24T23:11:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.921+0000 7f1241ffb700 1 -- 172.21.15.102:0/468725320 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1238009cf0 con 0x7f1244069f40 2024-04-24T23:11:18.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.922+0000 7f1241ffb700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f12340a2d30 0x7f12340a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:18.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.922+0000 7f1243fff700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f12340a2d30 0x7f12340a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:18.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.922+0000 7f1241ffb700 1 -- 172.21.15.102:0/468725320 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f12380cee60 con 0x7f1244069f40 2024-04-24T23:11:18.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.922+0000 7f1243fff700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f12340a2d30 0x7f12340a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f123000b5c0 tx=0x7f1230005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:18.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:18.926+0000 7f1241ffb700 1 -- 172.21.15.102:0/468725320 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1238020020 con 0x7f1244069f40 2024-04-24T23:11:19.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.125+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f124402db30 con 0x7f1244069f40 2024-04-24T23:11:19.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.127+0000 7f1241ffb700 1 -- 172.21.15.102:0/468725320 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22279 (secure 0 0 0) 0x7f123809be10 con 0x7f1244069f40 2024-04-24T23:11:19.129 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:11:19.129 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":86,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:11:02.189206+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f12340a2d30 msgr2=0x7f12340a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f12340a2d30 0x7f12340a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f123000b5c0 tx=0x7f1230005960 comp rx=0 tx=0).stop 2024-04-24T23:11:19.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 msgr2=0x7f1244101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 0x7f1244101200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f123800d980 tx=0x7f123800dc90 comp rx=0 tx=0).stop 2024-04-24T23:11:19.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 shutdown_connections 2024-04-24T23:11:19.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12441082b0 0x7f1244101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f12340a2d30 0x7f12340a51f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1244069f40 0x7f1244101200 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 --2- 172.21.15.102:0/468725320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124406a8e0 0x7f1244101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 >> 172.21.15.102:0/468725320 conn(0x7f12440780e0 msgr2=0x7f12440ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:19.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 shutdown_connections 2024-04-24T23:11:19.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:19.129+0000 7f124ae86700 1 -- 172.21.15.102:0/468725320 wait complete. 2024-04-24T23:11:19.504 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:19 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/468725320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:19.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:19 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/468725320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:19.852 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T23:11:19.853 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T23:11:19.853 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T23:11:19.854 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T23:11:19.854 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T23:11:19.855 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T23:11:19.855 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-24T23:11:19.855 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-24T23:11:19.856 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-24T23:11:19.857 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-24T23:11:19.857 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-24T23:11:19.858 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-24T23:11:19.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:19 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/468725320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:20.501 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:20.674 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:20 smithi102 ceph-mon[29216]: pgmap v302: 97 pgs: 97 active+clean; 625 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-04-24T23:11:20.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:20 smithi183 ceph-mon[33137]: pgmap v302: 97 pgs: 97 active+clean; 625 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-04-24T23:11:20.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:20 smithi161 ceph-mon[32709]: pgmap v302: 97 pgs: 97 active+clean; 625 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-04-24T23:11:21.316 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.317 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.317 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.318 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.319 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.326 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.326 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.331 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.334 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.338 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:21.340 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:23 smithi102 ceph-mon[29216]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-24T23:11:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:23 smithi161 ceph-mon[32709]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-24T23:11:23.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:23 smithi183 ceph-mon[33137]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-24T23:11:25.384 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:25 smithi102 ceph-mon[29216]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-24T23:11:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:25 smithi161 ceph-mon[32709]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-24T23:11:25.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 -- 172.21.15.102:0/637943648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0848102510 msgr2=0x7f08481028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 --2- 172.21.15.102:0/637943648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0848102510 0x7f08481028f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f08380099c0 tx=0x7f0838009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 -- 172.21.15.102:0/637943648 shutdown_connections 2024-04-24T23:11:25.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 --2- 172.21.15.102:0/637943648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08481075e0 0x7f084810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 --2- 172.21.15.102:0/637943648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0848102510 0x7f08481028f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 --2- 172.21.15.102:0/637943648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0848102ec0 0x7f0848106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 -- 172.21.15.102:0/637943648 >> 172.21.15.102:0/637943648 conn(0x7f08480793e0 msgr2=0x7f08480797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 -- 172.21.15.102:0/637943648 shutdown_connections 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 -- 172.21.15.102:0/637943648 wait complete. 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 Processor -- start 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.296+0000 7f084e59c700 1 -- start start 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f084e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0848102510 0x7f0848074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f084e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0848102ec0 0x7f0848074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f084e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08481075e0 0x7f0848074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f084e59c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f08481124a0 con 0x7f0848102510 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f084e59c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0848112320 con 0x7f08481075e0 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f084e59c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0848112620 con 0x7f0848102ec0 2024-04-24T23:11:25.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f0847fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0848102510 0x7f0848074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f0847fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0848102510 0x7f0848074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50396/0 (socket says 172.21.15.102:50396) 2024-04-24T23:11:25.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f0847fff700 1 -- 172.21.15.102:0/2529959549 learned_addr learned my addr 172.21.15.102:0/2529959549 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:25.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f084cb39700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08481075e0 0x7f0848074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f08477fe700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0848102ec0 0x7f0848074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f0847fff700 1 -- 172.21.15.102:0/2529959549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0848102ec0 msgr2=0x7f0848074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f0847fff700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0848102ec0 0x7f0848074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f0847fff700 1 -- 172.21.15.102:0/2529959549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08481075e0 msgr2=0x7f0848074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f0847fff700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08481075e0 0x7f0848074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f0847fff700 1 -- 172.21.15.102:0/2529959549 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08481a6e80 con 0x7f0848102510 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f084cb39700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08481075e0 0x7f0848074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.297+0000 7f0847fff700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0848102510 0x7f0848074530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f08380098e0 tx=0x7f0838005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.298+0000 7f08457fa700 1 -- 172.21.15.102:0/2529959549 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838027070 con 0x7f0848102510 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.298+0000 7f08457fa700 1 -- 172.21.15.102:0/2529959549 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0838004040 con 0x7f0848102510 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.298+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08481a7110 con 0x7f0848102510 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.298+0000 7f08457fa700 1 -- 172.21.15.102:0/2529959549 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838022e00 con 0x7f0848102510 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.298+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08481a7560 con 0x7f0848102510 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.299+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0848062900 con 0x7f0848102510 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.300+0000 7f08457fa700 1 -- 172.21.15.102:0/2529959549 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f083802a070 con 0x7f0848102510 2024-04-24T23:11:25.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.300+0000 7f08457fa700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f08300a2cb0 0x7f08300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.301+0000 7f08477fe700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f08300a2cb0 0x7f08300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.301+0000 7f08457fa700 1 -- 172.21.15.102:0/2529959549 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f08380d0250 con 0x7f0848102510 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.301+0000 7f08457fa700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] conn(0x7f08300ac490 0x7f08300ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.301+0000 7f08457fa700 1 -- 172.21.15.102:0/2529959549 --> [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0838001d30 con 0x7f08300ac490 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.301+0000 7f08457fa700 1 -- 172.21.15.102:0/2529959549 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f08380d0730 con 0x7f0848102510 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.301+0000 7f08477fe700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f08300a2cb0 0x7f08300a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f083c005ea0 tx=0x7f083c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.301+0000 7f084cb39700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] conn(0x7f08300ac490 0x7f08300ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.301+0000 7f084cb39700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] conn(0x7f08300ac490 0x7f08300ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.302+0000 7f08457fa700 1 -- 172.21.15.102:0/2529959549 <== osd.1 v2:172.21.15.161:6800/1328568264 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0838001d30 con 0x7f08300ac490 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.320+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 --> [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0848062d00 con 0x7f08300ac490 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.320+0000 7f08457fa700 1 -- 172.21.15.102:0/2529959549 <== osd.1 v2:172.21.15.161:6800/1328568264 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0848062d00 con 0x7f08300ac490 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 >> [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] conn(0x7f08300ac490 msgr2=0x7f08300ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] conn(0x7f08300ac490 0x7f08300ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f08300a2cb0 msgr2=0x7f08300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f08300a2cb0 0x7f08300a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f083c005ea0 tx=0x7f083c005e30 comp rx=0 tx=0).stop 2024-04-24T23:11:25.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0848102510 msgr2=0x7f0848074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 --2- 172.21.15.102:0/2529959549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0848102510 0x7f0848074530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f08380098e0 tx=0x7f0838005990 comp rx=0 tx=0).stop 2024-04-24T23:11:25.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084cb39700 1 -- 172.21.15.102:0/2529959549 reap_dead start 2024-04-24T23:11:25.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 shutdown_connections 2024-04-24T23:11:25.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 >> 172.21.15.102:0/2529959549 conn(0x7f08480793e0 msgr2=0x7f08481004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 shutdown_connections 2024-04-24T23:11:25.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:25.321+0000 7f084e59c700 1 -- 172.21.15.102:0/2529959549 wait complete. 2024-04-24T23:11:25.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:25 smithi183 ceph-mon[33137]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-24T23:11:27.536 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:27 smithi102 ceph-mon[29216]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-24T23:11:27.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:27 smithi183 ceph-mon[33137]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-24T23:11:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:27 smithi161 ceph-mon[32709]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-24T23:11:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:29 smithi102 ceph-mon[29216]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-24T23:11:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:29 smithi161 ceph-mon[32709]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-24T23:11:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:29 smithi183 ceph-mon[33137]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-24T23:11:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:31 smithi102 ceph-mon[29216]: pgmap v307: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-24T23:11:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:31 smithi161 ceph-mon[32709]: pgmap v307: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-24T23:11:31.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:31 smithi183 ceph-mon[33137]: pgmap v307: 97 pgs: 97 active+clean; 636 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-24T23:11:33.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.536+0000 7fc916a63700 1 -- 172.21.15.102:0/1519323333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc910102e90 msgr2=0x7fc910106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:33.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.536+0000 7fc916a63700 1 --2- 172.21.15.102:0/1519323333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc910102e90 0x7fc910106ec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc904009990 tx=0x7fc904009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.536+0000 7fc916a63700 1 -- 172.21.15.102:0/1519323333 shutdown_connections 2024-04-24T23:11:33.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.536+0000 7fc916a63700 1 --2- 172.21.15.102:0/1519323333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9101075a0 0x7fc91010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.536+0000 7fc916a63700 1 --2- 172.21.15.102:0/1519323333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9101024e0 0x7fc9101028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.536+0000 7fc916a63700 1 --2- 172.21.15.102:0/1519323333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc910102e90 0x7fc910106ec0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.536+0000 7fc916a63700 1 -- 172.21.15.102:0/1519323333 >> 172.21.15.102:0/1519323333 conn(0x7fc9100793e0 msgr2=0x7fc9100797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:33.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.536+0000 7fc916a63700 1 -- 172.21.15.102:0/1519323333 shutdown_connections 2024-04-24T23:11:33.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.536+0000 7fc916a63700 1 -- 172.21.15.102:0/1519323333 wait complete. 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc916a63700 1 Processor -- start 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc916a63700 1 -- start start 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc916a63700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9101024e0 0x7fc91019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc916a63700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc910102e90 0x7fc91019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc916a63700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9101075a0 0x7fc9101986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc916a63700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc910112410 con 0x7fc9101075a0 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc916a63700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc910112290 con 0x7fc9101024e0 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc916a63700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc910112590 con 0x7fc910102e90 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc915000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9101075a0 0x7fc9101986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:33.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc90f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc910102e90 0x7fc91019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc915000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9101075a0 0x7fc9101986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33004/0 (socket says 172.21.15.102:33004) 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc915000700 1 -- 172.21.15.102:0/1561279829 learned_addr learned my addr 172.21.15.102:0/1561279829 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.537+0000 7fc90ffff700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9101024e0 0x7fc91019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc915000700 1 -- 172.21.15.102:0/1561279829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc910102e90 msgr2=0x7fc91019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc915000700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc910102e90 0x7fc91019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc915000700 1 -- 172.21.15.102:0/1561279829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9101024e0 msgr2=0x7fc91019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc915000700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9101024e0 0x7fc91019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc915000700 1 -- 172.21.15.102:0/1561279829 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc910198fd0 con 0x7fc9101075a0 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc915000700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9101075a0 0x7fc9101986f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc90800cca0 tx=0x7fc90800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc90d7fa700 1 -- 172.21.15.102:0/1561279829 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908016020 con 0x7fc9101075a0 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9101992c0 con 0x7fc9101075a0 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc90d7fa700 1 -- 172.21.15.102:0/1561279829 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc90800aba0 con 0x7fc9101075a0 2024-04-24T23:11:33.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc90d7fa700 1 -- 172.21.15.102:0/1561279829 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080115e0 con 0x7fc9101075a0 2024-04-24T23:11:33.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.538+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9101a55a0 con 0x7fc9101075a0 2024-04-24T23:11:33.543 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:33 smithi102 ceph-mon[29216]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-24T23:11:33.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.539+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9100629c0 con 0x7fc9101075a0 2024-04-24T23:11:33.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.540+0000 7fc90d7fa700 1 -- 172.21.15.102:0/1561279829 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc908019020 con 0x7fc9101075a0 2024-04-24T23:11:33.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.541+0000 7fc90d7fa700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc8f80a2cb0 0x7fc8f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:33.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.541+0000 7fc90ffff700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc8f80a2cb0 0x7fc8f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:33.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.541+0000 7fc90d7fa700 1 -- 172.21.15.102:0/1561279829 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc9080cf210 con 0x7fc9101075a0 2024-04-24T23:11:33.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.541+0000 7fc90d7fa700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] conn(0x7fc8f80ac490 0x7fc8f80ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:33.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.541+0000 7fc90d7fa700 1 -- 172.21.15.102:0/1561279829 --> [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8f80aef60 con 0x7fc8f80ac490 2024-04-24T23:11:33.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.541+0000 7fc90d7fa700 1 -- 172.21.15.102:0/1561279829 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc9080cf6f0 con 0x7fc9101075a0 2024-04-24T23:11:33.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.541+0000 7fc90f7fe700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] conn(0x7fc8f80ac490 0x7fc8f80ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:33.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.541+0000 7fc90ffff700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc8f80a2cb0 0x7fc8f80a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc900005950 tx=0x7fc9000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:33.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.542+0000 7fc90f7fe700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] conn(0x7fc8f80ac490 0x7fc8f80ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:33.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.542+0000 7fc90d7fa700 1 -- 172.21.15.102:0/1561279829 <== osd.6 v2:172.21.15.102:6818/3573356846 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8f80aef60 con 0x7fc8f80ac490 2024-04-24T23:11:33.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.558+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 --> [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc910062e40 con 0x7fc8f80ac490 2024-04-24T23:11:33.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc90d7fa700 1 -- 172.21.15.102:0/1561279829 <== osd.6 v2:172.21.15.102:6818/3573356846 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc910062e40 con 0x7fc8f80ac490 2024-04-24T23:11:33.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] conn(0x7fc8f80ac490 msgr2=0x7fc8f80ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:33.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] conn(0x7fc8f80ac490 0x7fc8f80ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc8f80a2cb0 msgr2=0x7fc8f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:33.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc8f80a2cb0 0x7fc8f80a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc900005950 tx=0x7fc9000058e0 comp rx=0 tx=0).stop 2024-04-24T23:11:33.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9101075a0 msgr2=0x7fc9101986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:33.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 --2- 172.21.15.102:0/1561279829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9101075a0 0x7fc9101986f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc90800cca0 tx=0x7fc90800a060 comp rx=0 tx=0).stop 2024-04-24T23:11:33.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc915000700 1 -- 172.21.15.102:0/1561279829 reap_dead start 2024-04-24T23:11:33.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 shutdown_connections 2024-04-24T23:11:33.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 >> 172.21.15.102:0/1561279829 conn(0x7fc9100793e0 msgr2=0x7fc910101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:33.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 shutdown_connections 2024-04-24T23:11:33.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:33.559+0000 7fc916a63700 1 -- 172.21.15.102:0/1561279829 wait complete. 2024-04-24T23:11:33.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:33 smithi161 ceph-mon[32709]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-24T23:11:33.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:33 smithi183 ceph-mon[33137]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-24T23:11:34.664 INFO:teuthology.orchestra.run.smithi102.stdout:201863462959 2024-04-24T23:11:34.664 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-24T23:11:35.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:35 smithi102 ceph-mon[29216]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-24T23:11:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:35 smithi161 ceph-mon[32709]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-24T23:11:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:35 smithi183 ceph-mon[33137]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-24T23:11:36.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 -- 172.21.15.102:0/976183073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdad0100820 msgr2=0x7fdad0100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 --2- 172.21.15.102:0/976183073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdad0100820 0x7fdad0100c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009990 tx=0x7fdac0009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 -- 172.21.15.102:0/976183073 shutdown_connections 2024-04-24T23:11:36.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 --2- 172.21.15.102:0/976183073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad01011c0 0x7fdad010f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 --2- 172.21.15.102:0/976183073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad01082c0 0x7fdad01086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 --2- 172.21.15.102:0/976183073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdad0100820 0x7fdad0100c80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 -- 172.21.15.102:0/976183073 >> 172.21.15.102:0/976183073 conn(0x7fdad00777a0 msgr2=0x7fdad00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:36.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 -- 172.21.15.102:0/976183073 shutdown_connections 2024-04-24T23:11:36.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 -- 172.21.15.102:0/976183073 wait complete. 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.891+0000 7fdad7eea700 1 Processor -- start 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad7eea700 1 -- start start 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad7eea700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdad0100820 0x7fdad0101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad7eea700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad01011c0 0x7fdad0101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad7eea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad01082c0 0x7fdad0102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad7eea700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdad01123f0 con 0x7fdad0100820 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad7eea700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0112270 con 0x7fdad01082c0 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad7eea700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0112570 con 0x7fdad01011c0 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5c86700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdad0100820 0x7fdad0101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5c86700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdad0100820 0x7fdad0101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52060/0 (socket says 172.21.15.102:52060) 2024-04-24T23:11:36.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5c86700 1 -- 172.21.15.102:0/1754554556 learned_addr learned my addr 172.21.15.102:0/1754554556 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad6487700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad01082c0 0x7fdad0102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5485700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad01011c0 0x7fdad0101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5c86700 1 -- 172.21.15.102:0/1754554556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad01011c0 msgr2=0x7fdad0101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5c86700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdad01011c0 0x7fdad0101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5c86700 1 -- 172.21.15.102:0/1754554556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad01082c0 msgr2=0x7fdad0102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5c86700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad01082c0 0x7fdad0102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5c86700 1 -- 172.21.15.102:0/1754554556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad01a6d90 con 0x7fdad0100820 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad6487700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad01082c0 0x7fdad0102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdad5c86700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdad0100820 0x7fdad0101a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdac8007e20 tx=0x7fdac800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.892+0000 7fdac6ffd700 1 -- 172.21.15.102:0/1754554556 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8016020 con 0x7fdad0100820 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.893+0000 7fdac6ffd700 1 -- 172.21.15.102:0/1754554556 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdac800ae00 con 0x7fdad0100820 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.893+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad01a7080 con 0x7fdad0100820 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.893+0000 7fdac6ffd700 1 -- 172.21.15.102:0/1754554556 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8011700 con 0x7fdad0100820 2024-04-24T23:11:36.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.893+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad01a75c0 con 0x7fdad0100820 2024-04-24T23:11:36.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.893+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdad0062940 con 0x7fdad0100820 2024-04-24T23:11:36.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.895+0000 7fdac6ffd700 1 -- 172.21.15.102:0/1754554556 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdac8019020 con 0x7fdad0100820 2024-04-24T23:11:36.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.895+0000 7fdac6ffd700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdabc0a2cb0 0x7fdabc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.895+0000 7fdad5485700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdabc0a2cb0 0x7fdabc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.895+0000 7fdac6ffd700 1 -- 172.21.15.102:0/1754554556 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdac80cf490 con 0x7fdad0100820 2024-04-24T23:11:36.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.895+0000 7fdac6ffd700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] conn(0x7fdabc0ac490 0x7fdabc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.895+0000 7fdac6ffd700 1 -- 172.21.15.102:0/1754554556 --> [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdabc0aef60 con 0x7fdabc0ac490 2024-04-24T23:11:36.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.895+0000 7fdac6ffd700 1 -- 172.21.15.102:0/1754554556 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fdac80cf970 con 0x7fdad0100820 2024-04-24T23:11:36.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.896+0000 7fdad5485700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdabc0a2cb0 0x7fdabc0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdac0006010 tx=0x7fdac0005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:36.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.896+0000 7fdad6487700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] conn(0x7fdabc0ac490 0x7fdabc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.896+0000 7fdad6487700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] conn(0x7fdabc0ac490 0x7fdabc0ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:36.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.897+0000 7fdac6ffd700 1 -- 172.21.15.102:0/1754554556 <== osd.5 v2:172.21.15.183:6808/351285214 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdabc0aef60 con 0x7fdabc0ac490 2024-04-24T23:11:36.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.913+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 --> [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdad0062dc0 con 0x7fdabc0ac490 2024-04-24T23:11:36.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdac6ffd700 1 -- 172.21.15.102:0/1754554556 <== osd.5 v2:172.21.15.183:6808/351285214 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdad0062dc0 con 0x7fdabc0ac490 2024-04-24T23:11:36.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 >> [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] conn(0x7fdabc0ac490 msgr2=0x7fdabc0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] conn(0x7fdabc0ac490 0x7fdabc0ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdabc0a2cb0 msgr2=0x7fdabc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdabc0a2cb0 0x7fdabc0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdac0006010 tx=0x7fdac0005c40 comp rx=0 tx=0).stop 2024-04-24T23:11:36.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdad0100820 msgr2=0x7fdad0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 --2- 172.21.15.102:0/1754554556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdad0100820 0x7fdad0101a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdac8007e20 tx=0x7fdac800a340 comp rx=0 tx=0).stop 2024-04-24T23:11:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad6487700 1 -- 172.21.15.102:0/1754554556 reap_dead start 2024-04-24T23:11:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 shutdown_connections 2024-04-24T23:11:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 >> 172.21.15.102:0/1754554556 conn(0x7fdad00777a0 msgr2=0x7fdad00fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 shutdown_connections 2024-04-24T23:11:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.914+0000 7fdad7eea700 1 -- 172.21.15.102:0/1754554556 wait complete. 2024-04-24T23:11:36.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.234+0000 7f19b24bf700 1 -- 172.21.15.102:0/3960295437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19ac104e00 msgr2=0x7f19ac111310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.234+0000 7f19b24bf700 1 --2- 172.21.15.102:0/3960295437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19ac104e00 0x7f19ac111310 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1994009990 tx=0x7f1994009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.234+0000 7f19b24bf700 1 -- 172.21.15.102:0/3960295437 shutdown_connections 2024-04-24T23:11:36.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.234+0000 7f19b24bf700 1 --2- 172.21.15.102:0/3960295437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19ac104e00 0x7f19ac111310 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.234+0000 7f19b24bf700 1 --2- 172.21.15.102:0/3960295437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ac10a4d0 0x7f19ac10a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.234+0000 7f19b24bf700 1 --2- 172.21.15.102:0/3960295437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ac104440 0x7f19ac1048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.234+0000 7f19b24bf700 1 -- 172.21.15.102:0/3960295437 >> 172.21.15.102:0/3960295437 conn(0x7f19ac0fff30 msgr2=0x7f19ac102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.234+0000 7f19b24bf700 1 -- 172.21.15.102:0/3960295437 shutdown_connections 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.234+0000 7f19b24bf700 1 -- 172.21.15.102:0/3960295437 wait complete. 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19b24bf700 1 Processor -- start 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19b24bf700 1 -- start start 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19b24bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ac104440 0x7f19ac19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19b24bf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19ac104e00 0x7f19ac19b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19b24bf700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ac10a4d0 0x7f19ac19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19b24bf700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac1145e0 con 0x7f19ac104e00 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19b24bf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac114460 con 0x7f19ac104440 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19b24bf700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac114760 con 0x7f19ac10a4d0 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19ac104e00 0x7f19ac19b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19ac104e00 0x7f19ac19b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33008/0 (socket says 172.21.15.102:33008) 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19ab7fe700 1 -- 172.21.15.102:0/1520864555 learned_addr learned my addr 172.21.15.102:0/1520864555 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:36.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19abfff700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ac104440 0x7f19ac19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19ab7fe700 1 -- 172.21.15.102:0/1520864555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ac10a4d0 msgr2=0x7f19ac19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19ab7fe700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ac10a4d0 0x7f19ac19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19ab7fe700 1 -- 172.21.15.102:0/1520864555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ac104440 msgr2=0x7f19ac19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19ab7fe700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ac104440 0x7f19ac19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19ab7fe700 1 -- 172.21.15.102:0/1520864555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac19faf0 con 0x7f19ac104e00 2024-04-24T23:11:36.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19abfff700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ac104440 0x7f19ac19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:36.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.235+0000 7f19ab7fe700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19ac104e00 0x7f19ac19b030 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f19a40060b0 tx=0x7f19a4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:36.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.236+0000 7f19a97fa700 1 -- 172.21.15.102:0/1520864555 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a401a070 con 0x7f19ac104e00 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.236+0000 7f19a97fa700 1 -- 172.21.15.102:0/1520864555 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19a400de20 con 0x7f19ac104e00 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.236+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ac19fde0 con 0x7f19ac104e00 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.236+0000 7f19a97fa700 1 -- 172.21.15.102:0/1520864555 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4014720 con 0x7f19ac104e00 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.236+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ac1a76a0 con 0x7f19ac104e00 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.237+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19ac062940 con 0x7f19ac104e00 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.238+0000 7f19a97fa700 1 -- 172.21.15.102:0/1520864555 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f19a401d070 con 0x7f19ac104e00 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.238+0000 7f19a97fa700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19980a2cb0 0x7f19980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.238+0000 7f19abfff700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19980a2cb0 0x7f19980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.238+0000 7f19a97fa700 1 -- 172.21.15.102:0/1520864555 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f19a40cf230 con 0x7f19ac104e00 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.238+0000 7f19a97fa700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] conn(0x7f19980ac490 0x7f19980ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.238+0000 7f19a97fa700 1 -- 172.21.15.102:0/1520864555 --> [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19980aef60 con 0x7f19980ac490 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.238+0000 7f19a97fa700 1 -- 172.21.15.102:0/1520864555 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f19a40cf5e0 con 0x7f19ac104e00 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.239+0000 7f19abfff700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19980a2cb0 0x7f19980a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f199c005fd0 tx=0x7f199c005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:36.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.239+0000 7f19b0a5c700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] conn(0x7f19980ac490 0x7f19980ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.239+0000 7f19b0a5c700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] conn(0x7f19980ac490 0x7f19980ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.240+0000 7f19a97fa700 1 -- 172.21.15.102:0/1520864555 <== osd.0 v2:172.21.15.102:6802/825890094 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19980aef60 con 0x7f19980ac490 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 --> [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f19ac062d40 con 0x7f19980ac490 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19a97fa700 1 -- 172.21.15.102:0/1520864555 <== osd.0 v2:172.21.15.102:6802/825890094 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f19ac062d40 con 0x7f19980ac490 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] conn(0x7f19980ac490 msgr2=0x7f19980ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] conn(0x7f19980ac490 0x7f19980ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19980a2cb0 msgr2=0x7f19980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f19980a2cb0 0x7f19980a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f199c005fd0 tx=0x7f199c005d40 comp rx=0 tx=0).stop 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19ac104e00 msgr2=0x7f19ac19b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 --2- 172.21.15.102:0/1520864555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19ac104e00 0x7f19ac19b030 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f19a40060b0 tx=0x7f19a4007f90 comp rx=0 tx=0).stop 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b0a5c700 1 -- 172.21.15.102:0/1520864555 reap_dead start 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 shutdown_connections 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 >> 172.21.15.102:0/1520864555 conn(0x7f19ac0fff30 msgr2=0x7f19ac1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:36.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 shutdown_connections 2024-04-24T23:11:36.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:34.256+0000 7f19b24bf700 1 -- 172.21.15.102:0/1520864555 wait complete. 2024-04-24T23:11:37.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:37 smithi102 ceph-mon[29216]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T23:11:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:37 smithi161 ceph-mon[32709]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T23:11:37.733 INFO:teuthology.orchestra.run.smithi102.stdout:34359738445 2024-04-24T23:11:37.734 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T23:11:37.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:37 smithi183 ceph-mon[33137]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T23:11:37.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.285+0000 7fd986ac8700 1 -- 172.21.15.102:0/1625694525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980102f00 msgr2=0x7fd980106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.285+0000 7fd986ac8700 1 --2- 172.21.15.102:0/1625694525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980102f00 0x7fd980106e60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0099c0 tx=0x7fd96c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.285+0000 7fd986ac8700 1 -- 172.21.15.102:0/1625694525 shutdown_connections 2024-04-24T23:11:37.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.285+0000 7fd986ac8700 1 --2- 172.21.15.102:0/1625694525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd980107540 0x7fd98010f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.285+0000 7fd986ac8700 1 --2- 172.21.15.102:0/1625694525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd980102550 0x7fd980102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.285+0000 7fd986ac8700 1 --2- 172.21.15.102:0/1625694525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980102f00 0x7fd980106e60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.285+0000 7fd986ac8700 1 -- 172.21.15.102:0/1625694525 >> 172.21.15.102:0/1625694525 conn(0x7fd9800793f0 msgr2=0x7fd980079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.285+0000 7fd986ac8700 1 -- 172.21.15.102:0/1625694525 shutdown_connections 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.285+0000 7fd986ac8700 1 -- 172.21.15.102:0/1625694525 wait complete. 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd986ac8700 1 Processor -- start 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd986ac8700 1 -- start start 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd986ac8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd980102550 0x7fd980074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd986ac8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd980102f00 0x7fd980074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd986ac8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980107540 0x7fd980074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd986ac8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd980112500 con 0x7fd980107540 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd986ac8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd980112380 con 0x7fd980102f00 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd986ac8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd980112680 con 0x7fd980102550 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd985065700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980107540 0x7fd980074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:37.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd985065700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980107540 0x7fd980074fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33028/0 (socket says 172.21.15.102:33028) 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd985065700 1 -- 172.21.15.102:0/2333263612 learned_addr learned my addr 172.21.15.102:0/2333263612 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd97ffff700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd980102f00 0x7fd980074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd984864700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd980102550 0x7fd980074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd985065700 1 -- 172.21.15.102:0/2333263612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd980102550 msgr2=0x7fd980074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd985065700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd980102550 0x7fd980074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd985065700 1 -- 172.21.15.102:0/2333263612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd980102f00 msgr2=0x7fd980074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd985065700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd980102f00 0x7fd980074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.286+0000 7fd985065700 1 -- 172.21.15.102:0/2333263612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9801a6f90 con 0x7fd980107540 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.287+0000 7fd985065700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980107540 0x7fd980074fb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd97800b970 tx=0x7fd97800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.287+0000 7fd97dffb700 1 -- 172.21.15.102:0/2333263612 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97800d860 con 0x7fd980107540 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.287+0000 7fd97dffb700 1 -- 172.21.15.102:0/2333263612 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd97800e430 con 0x7fd980107540 2024-04-24T23:11:37.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.287+0000 7fd97dffb700 1 -- 172.21.15.102:0/2333263612 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978017660 con 0x7fd980107540 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.287+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9801a7280 con 0x7fd980107540 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.287+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9801a7790 con 0x7fd980107540 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.288+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd980062940 con 0x7fd980107540 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.289+0000 7fd97dffb700 1 -- 172.21.15.102:0/2333263612 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd97800e5e0 con 0x7fd980107540 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.289+0000 7fd97dffb700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd9700a2cb0 0x7fd9700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.289+0000 7fd984864700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd9700a2cb0 0x7fd9700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.290+0000 7fd97dffb700 1 -- 172.21.15.102:0/2333263612 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd9780cf1e0 con 0x7fd980107540 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.290+0000 7fd97dffb700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] conn(0x7fd9700ac490 0x7fd9700ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.290+0000 7fd97dffb700 1 -- 172.21.15.102:0/2333263612 --> [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd97801ce50 con 0x7fd9700ac490 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.290+0000 7fd97dffb700 1 -- 172.21.15.102:0/2333263612 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd9780cf6c0 con 0x7fd980107540 2024-04-24T23:11:37.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.290+0000 7fd97ffff700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] conn(0x7fd9700ac490 0x7fd9700ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:37.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.290+0000 7fd984864700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd9700a2cb0 0x7fd9700a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd974009a10 tx=0x7fd974009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:37.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.290+0000 7fd97ffff700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] conn(0x7fd9700ac490 0x7fd9700ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:37.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.291+0000 7fd97dffb700 1 -- 172.21.15.102:0/2333263612 <== osd.3 v2:172.21.15.102:6810/2459908351 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd97801ce50 con 0x7fd9700ac490 2024-04-24T23:11:37.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.307+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 --> [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd980062dc0 con 0x7fd9700ac490 2024-04-24T23:11:37.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.307+0000 7fd97dffb700 1 -- 172.21.15.102:0/2333263612 <== osd.3 v2:172.21.15.102:6810/2459908351 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd980062dc0 con 0x7fd9700ac490 2024-04-24T23:11:37.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.307+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] conn(0x7fd9700ac490 msgr2=0x7fd9700ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.307+0000 7fd986ac8700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] conn(0x7fd9700ac490 0x7fd9700ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.308+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd9700a2cb0 msgr2=0x7fd9700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.308+0000 7fd986ac8700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd9700a2cb0 0x7fd9700a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd974009a10 tx=0x7fd974009190 comp rx=0 tx=0).stop 2024-04-24T23:11:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.308+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980107540 msgr2=0x7fd980074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.308+0000 7fd986ac8700 1 --2- 172.21.15.102:0/2333263612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980107540 0x7fd980074fb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd97800b970 tx=0x7fd97800bc80 comp rx=0 tx=0).stop 2024-04-24T23:11:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.308+0000 7fd985065700 1 -- 172.21.15.102:0/2333263612 reap_dead start 2024-04-24T23:11:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.308+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 shutdown_connections 2024-04-24T23:11:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.308+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 >> 172.21.15.102:0/2333263612 conn(0x7fd9800793f0 msgr2=0x7fd980100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.308+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 shutdown_connections 2024-04-24T23:11:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:36.308+0000 7fd986ac8700 1 -- 172.21.15.102:0/2333263612 wait complete. 2024-04-24T23:11:38.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.220+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/645605285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ac102f00 msgr2=0x7fe9ac106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.220+0000 7fe9b3a9f700 1 --2- 172.21.15.102:0/645605285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ac102f00 0x7fe9ac106e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80099c0 tx=0x7fe9a8009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.220+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/645605285 shutdown_connections 2024-04-24T23:11:38.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.220+0000 7fe9b3a9f700 1 --2- 172.21.15.102:0/645605285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9ac107540 0x7fe9ac10f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.220+0000 7fe9b3a9f700 1 --2- 172.21.15.102:0/645605285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ac102550 0x7fe9ac102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.220+0000 7fe9b3a9f700 1 --2- 172.21.15.102:0/645605285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ac102f00 0x7fe9ac106e60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.220+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/645605285 >> 172.21.15.102:0/645605285 conn(0x7fe9ac0793f0 msgr2=0x7fe9ac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.220+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/645605285 shutdown_connections 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.220+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/645605285 wait complete. 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b3a9f700 1 Processor -- start 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b3a9f700 1 -- start start 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b3a9f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ac102550 0x7fe9ac19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b3a9f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ac102f00 0x7fe9ac19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b3a9f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9ac107540 0x7fe9ac198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b3a9f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac1124b0 con 0x7fe9ac102550 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b3a9f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac112330 con 0x7fe9ac102f00 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b3a9f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac112630 con 0x7fe9ac107540 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b183b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ac102550 0x7fe9ac19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b183b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ac102550 0x7fe9ac19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52070/0 (socket says 172.21.15.102:52070) 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b183b700 1 -- 172.21.15.102:0/1203389853 learned_addr learned my addr 172.21.15.102:0/1203389853 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:38.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.221+0000 7fe9b103a700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ac102f00 0x7fe9ac19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b203c700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9ac107540 0x7fe9ac198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b183b700 1 -- 172.21.15.102:0/1203389853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9ac107540 msgr2=0x7fe9ac198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b183b700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9ac107540 0x7fe9ac198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b183b700 1 -- 172.21.15.102:0/1203389853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ac102f00 msgr2=0x7fe9ac19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b183b700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ac102f00 0x7fe9ac19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b183b700 1 -- 172.21.15.102:0/1203389853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac1990d0 con 0x7fe9ac102550 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b203c700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9ac107540 0x7fe9ac198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b183b700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ac102550 0x7fe9ac19e760 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00b530 tx=0x7fe99c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9a2ffd700 1 -- 172.21.15.102:0/1203389853 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c015020 con 0x7fe9ac102550 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac1993c0 con 0x7fe9ac102550 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9a2ffd700 1 -- 172.21.15.102:0/1203389853 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe99c011ce0 con 0x7fe9ac102550 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac1a5630 con 0x7fe9ac102550 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.222+0000 7fe9a2ffd700 1 -- 172.21.15.102:0/1203389853 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c010d60 con 0x7fe9ac102550 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.223+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9ac062940 con 0x7fe9ac102550 2024-04-24T23:11:38.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.224+0000 7fe9a2ffd700 1 -- 172.21.15.102:0/1203389853 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe99c018020 con 0x7fe9ac102550 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.225+0000 7fe9a2ffd700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9980a2cb0 0x7fe9980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.225+0000 7fe9b103a700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9980a2cb0 0x7fe9980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.225+0000 7fe9a2ffd700 1 -- 172.21.15.102:0/1203389853 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe99c0cee80 con 0x7fe9ac102550 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.225+0000 7fe9a2ffd700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] conn(0x7fe9980ac490 0x7fe9980ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.225+0000 7fe9a2ffd700 1 -- 172.21.15.102:0/1203389853 --> [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe99c018ec0 con 0x7fe9980ac490 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.225+0000 7fe9a2ffd700 1 -- 172.21.15.102:0/1203389853 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe99c0cf360 con 0x7fe9ac102550 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.225+0000 7fe9b103a700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9980a2cb0 0x7fe9980a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8000c00 tx=0x7fe9a8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.225+0000 7fe9b203c700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] conn(0x7fe9980ac490 0x7fe9980ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.226+0000 7fe9b203c700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] conn(0x7fe9980ac490 0x7fe9980ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:38.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.227+0000 7fe9a2ffd700 1 -- 172.21.15.102:0/1203389853 <== osd.8 v2:172.21.15.183:6816/3691469531 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe99c018ec0 con 0x7fe9980ac490 2024-04-24T23:11:38.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.243+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 --> [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9ac062dc0 con 0x7fe9980ac490 2024-04-24T23:11:38.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.243+0000 7fe9a2ffd700 1 -- 172.21.15.102:0/1203389853 <== osd.8 v2:172.21.15.183:6816/3691469531 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe9ac062dc0 con 0x7fe9980ac490 2024-04-24T23:11:38.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 >> [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] conn(0x7fe9980ac490 msgr2=0x7fe9980ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] conn(0x7fe9980ac490 0x7fe9980ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9980a2cb0 msgr2=0x7fe9980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe9980a2cb0 0x7fe9980a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8000c00 tx=0x7fe9a8005fd0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ac102550 msgr2=0x7fe9ac19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 --2- 172.21.15.102:0/1203389853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ac102550 0x7fe9ac19e760 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00b530 tx=0x7fe99c00b8f0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b203c700 1 -- 172.21.15.102:0/1203389853 reap_dead start 2024-04-24T23:11:38.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 shutdown_connections 2024-04-24T23:11:38.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 >> 172.21.15.102:0/1203389853 conn(0x7fe9ac0793f0 msgr2=0x7fe9ac100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:38.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 shutdown_connections 2024-04-24T23:11:38.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.244+0000 7fe9b3a9f700 1 -- 172.21.15.102:0/1203389853 wait complete. 2024-04-24T23:11:38.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.433+0000 7f8a93818700 1 -- 172.21.15.102:0/2735746135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a8c102540 msgr2=0x7f8a8c105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.433+0000 7f8a93818700 1 --2- 172.21.15.102:0/2735746135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a8c102540 0x7f8a8c105dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8a880099c0 tx=0x7f8a88009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.433+0000 7f8a93818700 1 -- 172.21.15.102:0/2735746135 shutdown_connections 2024-04-24T23:11:38.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.433+0000 7f8a93818700 1 --2- 172.21.15.102:0/2735746135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a8c1064b0 0x7f8a8c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.433+0000 7f8a93818700 1 --2- 172.21.15.102:0/2735746135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a8c101b90 0x7f8a8c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.433+0000 7f8a93818700 1 --2- 172.21.15.102:0/2735746135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a8c102540 0x7f8a8c105dd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.433+0000 7f8a93818700 1 -- 172.21.15.102:0/2735746135 >> 172.21.15.102:0/2735746135 conn(0x7f8a8c0777a0 msgr2=0x7f8a8c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.433+0000 7f8a93818700 1 -- 172.21.15.102:0/2735746135 shutdown_connections 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.433+0000 7f8a93818700 1 -- 172.21.15.102:0/2735746135 wait complete. 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a93818700 1 Processor -- start 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a93818700 1 -- start start 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a93818700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a8c101b90 0x7f8a8c0fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a93818700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a8c102540 0x7f8a8c0fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a93818700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a8c1064b0 0x7f8a8c0fd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a93818700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1113e0 con 0x7f8a8c1064b0 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a93818700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c111260 con 0x7f8a8c102540 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a93818700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c111560 con 0x7f8a8c101b90 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a91db5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a8c1064b0 0x7f8a8c0fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a91db5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a8c1064b0 0x7f8a8c0fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52094/0 (socket says 172.21.15.102:52094) 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a91db5700 1 -- 172.21.15.102:0/2629461978 learned_addr learned my addr 172.21.15.102:0/2629461978 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a915b4700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a8c101b90 0x7f8a8c0fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.434+0000 7f8a90db3700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a8c102540 0x7f8a8c0fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a91db5700 1 -- 172.21.15.102:0/2629461978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a8c101b90 msgr2=0x7f8a8c0fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a91db5700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a8c101b90 0x7f8a8c0fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a91db5700 1 -- 172.21.15.102:0/2629461978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a8c102540 msgr2=0x7f8a8c0fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a91db5700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a8c102540 0x7f8a8c0fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a91db5700 1 -- 172.21.15.102:0/2629461978 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c1a6f10 con 0x7f8a8c1064b0 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a91db5700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a8c1064b0 0x7f8a8c0fd690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007d70 tx=0x7f8a8400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a827fc700 1 -- 172.21.15.102:0/2629461978 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84016020 con 0x7f8a8c1064b0 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a827fc700 1 -- 172.21.15.102:0/2629461978 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a8400ac90 con 0x7f8a8c1064b0 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8c1a7200 con 0x7f8a8c1064b0 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a827fc700 1 -- 172.21.15.102:0/2629461978 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a840115e0 con 0x7f8a8c1064b0 2024-04-24T23:11:38.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.435+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8c1a76c0 con 0x7f8a8c1064b0 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.436+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a8c105660 con 0x7f8a8c1064b0 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.437+0000 7f8a827fc700 1 -- 172.21.15.102:0/2629461978 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8a84019020 con 0x7f8a8c1064b0 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.437+0000 7f8a827fc700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8a780a2cb0 0x7f8a780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.438+0000 7f8a915b4700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8a780a2cb0 0x7f8a780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.438+0000 7f8a827fc700 1 -- 172.21.15.102:0/2629461978 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8a840cf3c0 con 0x7f8a8c1064b0 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.438+0000 7f8a827fc700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] conn(0x7f8a780ac490 0x7f8a780ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.438+0000 7f8a827fc700 1 -- 172.21.15.102:0/2629461978 --> [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a8401ce50 con 0x7f8a780ac490 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.438+0000 7f8a827fc700 1 -- 172.21.15.102:0/2629461978 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8a840cf8a0 con 0x7f8a8c1064b0 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.438+0000 7f8a915b4700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8a780a2cb0 0x7f8a780a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009a10 tx=0x7f8a7c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.438+0000 7f8a90db3700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] conn(0x7f8a780ac490 0x7f8a780ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.439+0000 7f8a90db3700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] conn(0x7f8a780ac490 0x7f8a780ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:38.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.439+0000 7f8a827fc700 1 -- 172.21.15.102:0/2629461978 <== osd.11 v2:172.21.15.183:6824/2953790548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a8401ce50 con 0x7f8a780ac490 2024-04-24T23:11:38.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.456+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 --> [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a8c068f20 con 0x7f8a780ac490 2024-04-24T23:11:38.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.456+0000 7f8a827fc700 1 -- 172.21.15.102:0/2629461978 <== osd.11 v2:172.21.15.183:6824/2953790548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a8c068f20 con 0x7f8a780ac490 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 >> [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] conn(0x7f8a780ac490 msgr2=0x7f8a780ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] conn(0x7f8a780ac490 0x7f8a780ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8a780a2cb0 msgr2=0x7f8a780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8a780a2cb0 0x7f8a780a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009a10 tx=0x7f8a7c009190 comp rx=0 tx=0).stop 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a8c1064b0 msgr2=0x7f8a8c0fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 --2- 172.21.15.102:0/2629461978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a8c1064b0 0x7f8a8c0fd690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007d70 tx=0x7f8a8400a340 comp rx=0 tx=0).stop 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a91db5700 1 -- 172.21.15.102:0/2629461978 reap_dead start 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 shutdown_connections 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 >> 172.21.15.102:0/2629461978 conn(0x7f8a8c0777a0 msgr2=0x7f8a8c100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 shutdown_connections 2024-04-24T23:11:38.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.457+0000 7f8a93818700 1 -- 172.21.15.102:0/2629461978 wait complete. 2024-04-24T23:11:39.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:39 smithi102 ceph-mon[29216]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T23:11:39.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:39 smithi161 ceph-mon[32709]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T23:11:39.622 INFO:teuthology.orchestra.run.smithi102.stdout:167503724597 2024-04-24T23:11:39.623 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T23:11:39.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:39 smithi183 ceph-mon[33137]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T23:11:40.577 INFO:teuthology.orchestra.run.smithi102.stdout:347892350998 2024-04-24T23:11:40.577 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-24T23:11:40.796 INFO:teuthology.orchestra.run.smithi102.stdout:55834574918 2024-04-24T23:11:40.797 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T23:11:41.423 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:41 smithi102 ceph-mon[29216]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-24T23:11:41.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:41 smithi161 ceph-mon[32709]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-24T23:11:41.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:41 smithi183 ceph-mon[33137]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-24T23:11:41.780 INFO:teuthology.orchestra.run.smithi102.stdout:257698037797 2024-04-24T23:11:41.781 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-24T23:11:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.652+0000 7f5fa760c700 1 -- 172.21.15.102:0/2813374501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fa01071a0 msgr2=0x7f5fa0107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.652+0000 7f5fa760c700 1 --2- 172.21.15.102:0/2813374501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fa01071a0 0x7f5fa0107580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009990 tx=0x7f5f90009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.652+0000 7f5fa760c700 1 -- 172.21.15.102:0/2813374501 shutdown_connections 2024-04-24T23:11:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.652+0000 7f5fa760c700 1 --2- 172.21.15.102:0/2813374501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa0101af0 0x7f5fa010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.652+0000 7f5fa760c700 1 --2- 172.21.15.102:0/2813374501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fa01071a0 0x7f5fa0107580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.652+0000 7f5fa760c700 1 --2- 172.21.15.102:0/2813374501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fa0101130 0x7f5fa01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.652+0000 7f5fa760c700 1 -- 172.21.15.102:0/2813374501 >> 172.21.15.102:0/2813374501 conn(0x7f5fa00780e0 msgr2=0x7f5fa00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.652+0000 7f5fa760c700 1 -- 172.21.15.102:0/2813374501 shutdown_connections 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.652+0000 7f5fa760c700 1 -- 172.21.15.102:0/2813374501 wait complete. 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa760c700 1 Processor -- start 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa760c700 1 -- start start 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa760c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fa0101130 0x7f5fa0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa760c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fa0101af0 0x7f5fa0199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa760c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa01071a0 0x7f5fa01a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa760c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01112c0 con 0x7f5fa0101130 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa760c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0111140 con 0x7f5fa0101af0 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa760c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0111440 con 0x7f5fa01071a0 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa53a8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fa0101130 0x7f5fa0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa53a8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fa0101130 0x7f5fa0199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52118/0 (socket says 172.21.15.102:52118) 2024-04-24T23:11:42.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa53a8700 1 -- 172.21.15.102:0/2038640587 learned_addr learned my addr 172.21.15.102:0/2038640587 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:42.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.653+0000 7f5fa5ba9700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa01071a0 0x7f5fa01a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:42.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5fa53a8700 1 -- 172.21.15.102:0/2038640587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa01071a0 msgr2=0x7f5fa01a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5fa53a8700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa01071a0 0x7f5fa01a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5fa53a8700 1 -- 172.21.15.102:0/2038640587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fa0101af0 msgr2=0x7f5fa0199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5fa53a8700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fa0101af0 0x7f5fa0199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5fa53a8700 1 -- 172.21.15.102:0/2038640587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa01a4d00 con 0x7f5fa0101130 2024-04-24T23:11:42.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5fa5ba9700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fa01071a0 0x7f5fa01a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:42.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5fa53a8700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fa0101130 0x7f5fa0199740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f90003ab0 tx=0x7f5f90003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:42.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5f967fc700 1 -- 172.21.15.102:0/2038640587 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90027070 con 0x7f5fa0101130 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa01a4ed0 con 0x7f5fa0101130 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5f967fc700 1 -- 172.21.15.102:0/2038640587 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f9000bdc0 con 0x7f5fa0101130 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5f967fc700 1 -- 172.21.15.102:0/2038640587 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90022d40 con 0x7f5fa0101130 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.654+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa01a5330 con 0x7f5fa0101130 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.655+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5fa0062940 con 0x7f5fa0101130 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.656+0000 7f5f967fc700 1 -- 172.21.15.102:0/2038640587 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5f9002a070 con 0x7f5fa0101130 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.657+0000 7f5f967fc700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5f8c0a2cb0 0x7f5f8c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.657+0000 7f5fa4ba7700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5f8c0a2cb0 0x7f5f8c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.657+0000 7f5f967fc700 1 -- 172.21.15.102:0/2038640587 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5f900d08c0 con 0x7f5fa0101130 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.657+0000 7f5f967fc700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] conn(0x7f5f8c0ac490 0x7f5f8c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.657+0000 7f5f967fc700 1 -- 172.21.15.102:0/2038640587 --> [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f9000b0b0 con 0x7f5f8c0ac490 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.657+0000 7f5f967fc700 1 -- 172.21.15.102:0/2038640587 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5f900d0da0 con 0x7f5fa0101130 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.657+0000 7f5fa4ba7700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5f8c0a2cb0 0x7f5f8c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005fd0 tx=0x7f5f9c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:42.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.657+0000 7f5fa5ba9700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] conn(0x7f5f8c0ac490 0x7f5f8c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.658+0000 7f5fa5ba9700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] conn(0x7f5f8c0ac490 0x7f5f8c0ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.658+0000 7f5f967fc700 1 -- 172.21.15.102:0/2038640587 <== osd.2 v2:172.21.15.183:6800/1173587665 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f9000b0b0 con 0x7f5f8c0ac490 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.674+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 --> [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5fa0062d40 con 0x7f5f8c0ac490 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.675+0000 7f5f967fc700 1 -- 172.21.15.102:0/2038640587 <== osd.2 v2:172.21.15.183:6800/1173587665 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5fa0062d40 con 0x7f5f8c0ac490 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.675+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 >> [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] conn(0x7f5f8c0ac490 msgr2=0x7f5f8c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.675+0000 7f5fa760c700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] conn(0x7f5f8c0ac490 0x7f5f8c0ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.675+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5f8c0a2cb0 msgr2=0x7f5f8c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.675+0000 7f5fa760c700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5f8c0a2cb0 0x7f5f8c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005fd0 tx=0x7f5f9c005d00 comp rx=0 tx=0).stop 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.675+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fa0101130 msgr2=0x7f5fa0199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.675+0000 7f5fa760c700 1 --2- 172.21.15.102:0/2038640587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fa0101130 0x7f5fa0199740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f90003ab0 tx=0x7f5f90003b90 comp rx=0 tx=0).stop 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.675+0000 7f5fa5ba9700 1 -- 172.21.15.102:0/2038640587 reap_dead start 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.676+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 shutdown_connections 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.676+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 >> 172.21.15.102:0/2038640587 conn(0x7f5fa00780e0 msgr2=0x7f5fa00ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.676+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 shutdown_connections 2024-04-24T23:11:42.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:38.676+0000 7f5fa760c700 1 -- 172.21.15.102:0/2038640587 wait complete. 2024-04-24T23:11:42.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.502+0000 7fe4dd183700 1 -- 172.21.15.102:0/4244631895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d8102540 msgr2=0x7fe4d8102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.502+0000 7fe4dd183700 1 --2- 172.21.15.102:0/4244631895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d8102540 0x7fe4d8102920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009990 tx=0x7fe4d0009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.502+0000 7fe4dd183700 1 -- 172.21.15.102:0/4244631895 shutdown_connections 2024-04-24T23:11:42.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.502+0000 7fe4dd183700 1 --2- 172.21.15.102:0/4244631895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4d81075a0 0x7fe4d810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.502+0000 7fe4dd183700 1 --2- 172.21.15.102:0/4244631895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d8102540 0x7fe4d8102920 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.502+0000 7fe4dd183700 1 --2- 172.21.15.102:0/4244631895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d8102ef0 0x7fe4d8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.502+0000 7fe4dd183700 1 -- 172.21.15.102:0/4244631895 >> 172.21.15.102:0/4244631895 conn(0x7fe4d8079310 msgr2=0x7fe4d8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:42.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.502+0000 7fe4dd183700 1 -- 172.21.15.102:0/4244631895 shutdown_connections 2024-04-24T23:11:42.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.502+0000 7fe4dd183700 1 -- 172.21.15.102:0/4244631895 wait complete. 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4dd183700 1 Processor -- start 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4dd183700 1 -- start start 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4dd183700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d8102540 0x7fe4d819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4dd183700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d8102ef0 0x7fe4d819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4dd183700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4d81075a0 0x7fe4d8198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4dd183700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8112510 con 0x7fe4d8102ef0 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4dd183700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8112390 con 0x7fe4d81075a0 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4dd183700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8112690 con 0x7fe4d8102540 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4d659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d8102ef0 0x7fe4d819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4d659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d8102ef0 0x7fe4d819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52130/0 (socket says 172.21.15.102:52130) 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4d659c700 1 -- 172.21.15.102:0/3336170787 learned_addr learned my addr 172.21.15.102:0/3336170787 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4d759e700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4d81075a0 0x7fe4d8198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:42.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4d6d9d700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d8102540 0x7fe4d819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4d659c700 1 -- 172.21.15.102:0/3336170787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d8102540 msgr2=0x7fe4d819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4d659c700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d8102540 0x7fe4d819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4d659c700 1 -- 172.21.15.102:0/3336170787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4d81075a0 msgr2=0x7fe4d8198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.503+0000 7fe4d659c700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4d81075a0 0x7fe4d8198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.504+0000 7fe4d659c700 1 -- 172.21.15.102:0/3336170787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d8198ff0 con 0x7fe4d8102ef0 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.504+0000 7fe4d659c700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d8102ef0 0x7fe4d819ebd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00b9c0 tx=0x7fe4cc00bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.504+0000 7fe4bffff700 1 -- 172.21.15.102:0/3336170787 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00d8e0 con 0x7fe4d8102ef0 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.504+0000 7fe4bffff700 1 -- 172.21.15.102:0/3336170787 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4cc015420 con 0x7fe4d8102ef0 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.504+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d81992e0 con 0x7fe4d8102ef0 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.504+0000 7fe4bffff700 1 -- 172.21.15.102:0/3336170787 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00c660 con 0x7fe4d8102ef0 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.504+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d8074460 con 0x7fe4d8102ef0 2024-04-24T23:11:42.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.505+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4d80629c0 con 0x7fe4d8102ef0 2024-04-24T23:11:42.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.506+0000 7fe4bffff700 1 -- 172.21.15.102:0/3336170787 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe4cc0155d0 con 0x7fe4d8102ef0 2024-04-24T23:11:42.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.507+0000 7fe4bffff700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe4c00a2cb0 0x7fe4c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.507+0000 7fe4d6d9d700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe4c00a2cb0 0x7fe4c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:42.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.507+0000 7fe4bffff700 1 -- 172.21.15.102:0/3336170787 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe4cc0cf510 con 0x7fe4d8102ef0 2024-04-24T23:11:42.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.507+0000 7fe4bffff700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] conn(0x7fe4c00ac490 0x7fe4c00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:42.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.507+0000 7fe4bffff700 1 -- 172.21.15.102:0/3336170787 --> [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4cc019e70 con 0x7fe4c00ac490 2024-04-24T23:11:42.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.507+0000 7fe4bffff700 1 -- 172.21.15.102:0/3336170787 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe4cc0cf8c0 con 0x7fe4d8102ef0 2024-04-24T23:11:42.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.507+0000 7fe4d6d9d700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe4c00a2cb0 0x7fe4c00a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000b5c0 tx=0x7fe4d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:42.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.507+0000 7fe4d759e700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] conn(0x7fe4c00ac490 0x7fe4c00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:42.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.508+0000 7fe4d759e700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] conn(0x7fe4c00ac490 0x7fe4c00ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:42.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.509+0000 7fe4bffff700 1 -- 172.21.15.102:0/3336170787 <== osd.7 v2:172.21.15.161:6816/1449722417 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4cc019e70 con 0x7fe4c00ac490 2024-04-24T23:11:42.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.526+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 --> [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4d8062df0 con 0x7fe4c00ac490 2024-04-24T23:11:42.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.526+0000 7fe4bffff700 1 -- 172.21.15.102:0/3336170787 <== osd.7 v2:172.21.15.161:6816/1449722417 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe4d8062df0 con 0x7fe4c00ac490 2024-04-24T23:11:42.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 >> [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] conn(0x7fe4c00ac490 msgr2=0x7fe4c00ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] conn(0x7fe4c00ac490 0x7fe4c00ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe4c00a2cb0 msgr2=0x7fe4c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe4c00a2cb0 0x7fe4c00a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000b5c0 tx=0x7fe4d0005960 comp rx=0 tx=0).stop 2024-04-24T23:11:42.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d8102ef0 msgr2=0x7fe4d819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:42.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 --2- 172.21.15.102:0/3336170787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d8102ef0 0x7fe4d819ebd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00b9c0 tx=0x7fe4cc00bcd0 comp rx=0 tx=0).stop 2024-04-24T23:11:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4d759e700 1 -- 172.21.15.102:0/3336170787 reap_dead start 2024-04-24T23:11:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 shutdown_connections 2024-04-24T23:11:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 >> 172.21.15.102:0/3336170787 conn(0x7fe4d8079310 msgr2=0x7fe4d8100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 shutdown_connections 2024-04-24T23:11:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:42.527+0000 7fe4dd183700 1 -- 172.21.15.102:0/3336170787 wait complete. 2024-04-24T23:11:43.184 INFO:teuthology.orchestra.run.smithi102.stdout:77309411396 2024-04-24T23:11:43.184 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T23:11:43.488 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:43 smithi102 ceph-mon[29216]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-24T23:11:43.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:43 smithi161 ceph-mon[32709]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-24T23:11:43.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:43 smithi183 ceph-mon[33137]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.006+0000 7fdb001f3700 1 -- 172.21.15.102:0/2281813468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81022e0 msgr2=0x7fdaf8102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.006+0000 7fdb001f3700 1 --2- 172.21.15.102:0/2281813468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81022e0 0x7fdaf8102760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdae80099c0 tx=0x7fdae8009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.006+0000 7fdb001f3700 1 -- 172.21.15.102:0/2281813468 shutdown_connections 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.006+0000 7fdb001f3700 1 --2- 172.21.15.102:0/2281813468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaf8102ca0 0x7fdaf810f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.006+0000 7fdb001f3700 1 --2- 172.21.15.102:0/2281813468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf81082f0 0x7fdaf81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.006+0000 7fdb001f3700 1 --2- 172.21.15.102:0/2281813468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81022e0 0x7fdaf8102760 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.006+0000 7fdb001f3700 1 -- 172.21.15.102:0/2281813468 >> 172.21.15.102:0/2281813468 conn(0x7fdaf8079390 msgr2=0x7fdaf80797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.006+0000 7fdb001f3700 1 -- 172.21.15.102:0/2281813468 shutdown_connections 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.006+0000 7fdb001f3700 1 -- 172.21.15.102:0/2281813468 wait complete. 2024-04-24T23:11:44.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdb001f3700 1 Processor -- start 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdb001f3700 1 -- start start 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdb001f3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81022e0 0x7fdaf819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdb001f3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf8102ca0 0x7fdaf819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdb001f3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaf81082f0 0x7fdaf81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdb001f3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8112510 con 0x7fdaf81022e0 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdb001f3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8112390 con 0x7fdaf81082f0 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdb001f3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8112690 con 0x7fdaf8102ca0 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafdf8f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81022e0 0x7fdaf819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafdf8f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81022e0 0x7fdaf819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52146/0 (socket says 172.21.15.102:52146) 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafdf8f700 1 -- 172.21.15.102:0/2455592163 learned_addr learned my addr 172.21.15.102:0/2455592163 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafd78e700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf8102ca0 0x7fdaf819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafe790700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaf81082f0 0x7fdaf81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafdf8f700 1 -- 172.21.15.102:0/2455592163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf8102ca0 msgr2=0x7fdaf819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafdf8f700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf8102ca0 0x7fdaf819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafdf8f700 1 -- 172.21.15.102:0/2455592163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaf81082f0 msgr2=0x7fdaf81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafdf8f700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaf81082f0 0x7fdaf81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.007+0000 7fdafdf8f700 1 -- 172.21.15.102:0/2455592163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf8198fd0 con 0x7fdaf81022e0 2024-04-24T23:11:44.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.008+0000 7fdafdf8f700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81022e0 0x7fdaf819e670 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400c970 tx=0x7fdaf400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:44.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.008+0000 7fdaeeffd700 1 -- 172.21.15.102:0/2455592163 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4007aa0 con 0x7fdaf81022e0 2024-04-24T23:11:44.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.008+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf81992c0 con 0x7fdaf81022e0 2024-04-24T23:11:44.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.008+0000 7fdaeeffd700 1 -- 172.21.15.102:0/2455592163 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaf4004ce0 con 0x7fdaf81022e0 2024-04-24T23:11:44.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.008+0000 7fdaeeffd700 1 -- 172.21.15.102:0/2455592163 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4005520 con 0x7fdaf81022e0 2024-04-24T23:11:44.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.008+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf8074460 con 0x7fdaf81022e0 2024-04-24T23:11:44.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.009+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdaf80629c0 con 0x7fdaf81022e0 2024-04-24T23:11:44.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.010+0000 7fdaeeffd700 1 -- 172.21.15.102:0/2455592163 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdaf4004750 con 0x7fdaf81022e0 2024-04-24T23:11:44.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.010+0000 7fdaeeffd700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdae40a2cb0 0x7fdae40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.011+0000 7fdafd78e700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdae40a2cb0 0x7fdae40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.011+0000 7fdaeeffd700 1 -- 172.21.15.102:0/2455592163 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdaf40cf2c0 con 0x7fdaf81022e0 2024-04-24T23:11:44.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.011+0000 7fdaeeffd700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] conn(0x7fdae40ac490 0x7fdae40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.011+0000 7fdaeeffd700 1 -- 172.21.15.102:0/2455592163 --> [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdaf400eec0 con 0x7fdae40ac490 2024-04-24T23:11:44.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.011+0000 7fdaeeffd700 1 -- 172.21.15.102:0/2455592163 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fdaf40cf7a0 con 0x7fdaf81022e0 2024-04-24T23:11:44.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.011+0000 7fdafd78e700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdae40a2cb0 0x7fdae40a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8079a40 tx=0x7fdae8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:44.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.011+0000 7fdafe790700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] conn(0x7fdae40ac490 0x7fdae40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.011+0000 7fdafe790700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] conn(0x7fdae40ac490 0x7fdae40ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:44.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.012+0000 7fdaeeffd700 1 -- 172.21.15.102:0/2455592163 <== osd.10 v2:172.21.15.161:6824/968108427 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdaf400eec0 con 0x7fdae40ac490 2024-04-24T23:11:44.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.034+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 --> [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdaf8062e40 con 0x7fdae40ac490 2024-04-24T23:11:44.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.034+0000 7fdaeeffd700 1 -- 172.21.15.102:0/2455592163 <== osd.10 v2:172.21.15.161:6824/968108427 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdaf8062e40 con 0x7fdae40ac490 2024-04-24T23:11:44.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 >> [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] conn(0x7fdae40ac490 msgr2=0x7fdae40ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] conn(0x7fdae40ac490 0x7fdae40ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdae40a2cb0 msgr2=0x7fdae40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdae40a2cb0 0x7fdae40a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8079a40 tx=0x7fdae8005fb0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81022e0 msgr2=0x7fdaf819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 --2- 172.21.15.102:0/2455592163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81022e0 0x7fdaf819e670 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400c970 tx=0x7fdaf400cc80 comp rx=0 tx=0).stop 2024-04-24T23:11:44.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdafe790700 1 -- 172.21.15.102:0/2455592163 reap_dead start 2024-04-24T23:11:44.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 shutdown_connections 2024-04-24T23:11:44.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 >> 172.21.15.102:0/2455592163 conn(0x7fdaf8079390 msgr2=0x7fdaf8101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:44.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 shutdown_connections 2024-04-24T23:11:44.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.035+0000 7fdb001f3700 1 -- 172.21.15.102:0/2455592163 wait complete. 2024-04-24T23:11:44.454 INFO:teuthology.orchestra.run.smithi102.stdout:223338299435 2024-04-24T23:11:44.455 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-24T23:11:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.744+0000 7f7524551700 1 -- 172.21.15.102:0/992604907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f751c100780 msgr2=0x7f751c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.744+0000 7f7524551700 1 --2- 172.21.15.102:0/992604907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f751c100780 0x7f751c100c00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f750c0099c0 tx=0x7f750c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 -- 172.21.15.102:0/992604907 shutdown_connections 2024-04-24T23:11:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 --2- 172.21.15.102:0/992604907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c101140 0x7f751c10f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 --2- 172.21.15.102:0/992604907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751c107250 0x7f751c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 --2- 172.21.15.102:0/992604907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f751c100780 0x7f751c100c00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 -- 172.21.15.102:0/992604907 >> 172.21.15.102:0/992604907 conn(0x7f751c077730 msgr2=0x7f751c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:44.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 -- 172.21.15.102:0/992604907 shutdown_connections 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 -- 172.21.15.102:0/992604907 wait complete. 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 Processor -- start 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 -- start start 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f751c100780 0x7f751c101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c101140 0x7f751c101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751c107250 0x7f751c1023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f751c112580 con 0x7f751c100780 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f751c112400 con 0x7f751c101140 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f7524551700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f751c112700 con 0x7f751c107250 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.745+0000 7f75222ed700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f751c100780 0x7f751c101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75222ed700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f751c100780 0x7f751c101960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52164/0 (socket says 172.21.15.102:52164) 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75222ed700 1 -- 172.21.15.102:0/69737520 learned_addr learned my addr 172.21.15.102:0/69737520 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f7522aee700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751c107250 0x7f751c1023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f7521aec700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c101140 0x7f751c101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75222ed700 1 -- 172.21.15.102:0/69737520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751c107250 msgr2=0x7f751c1023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75222ed700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f751c107250 0x7f751c1023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75222ed700 1 -- 172.21.15.102:0/69737520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c101140 msgr2=0x7f751c101ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75222ed700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c101140 0x7f751c101ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75222ed700 1 -- 172.21.15.102:0/69737520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c1a6f10 con 0x7f751c100780 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75222ed700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f751c100780 0x7f751c101960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f751800c970 tx=0x7f751800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75137fe700 1 -- 172.21.15.102:0/69737520 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518007aa0 con 0x7f751c100780 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c1a7200 con 0x7f751c100780 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75137fe700 1 -- 172.21.15.102:0/69737520 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7518004ce0 con 0x7f751c100780 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f75137fe700 1 -- 172.21.15.102:0/69737520 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518005520 con 0x7f751c100780 2024-04-24T23:11:44.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.746+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c1a7710 con 0x7f751c100780 2024-04-24T23:11:44.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.747+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f751c062940 con 0x7f751c100780 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.748+0000 7f75137fe700 1 -- 172.21.15.102:0/69737520 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7518004750 con 0x7f751c100780 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.749+0000 7f75137fe700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75080a2d30 0x7f75080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.749+0000 7f7521aec700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75080a2d30 0x7f75080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.749+0000 7f75137fe700 1 -- 172.21.15.102:0/69737520 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f75180cf2a0 con 0x7f751c100780 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.749+0000 7f75137fe700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] conn(0x7f75080ac510 0x7f75080ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.749+0000 7f75137fe700 1 -- 172.21.15.102:0/69737520 --> [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7518013e80 con 0x7f75080ac510 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.749+0000 7f75137fe700 1 -- 172.21.15.102:0/69737520 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f75180cf780 con 0x7f751c100780 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.749+0000 7f7522aee700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] conn(0x7f75080ac510 0x7f75080ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.750+0000 7f7521aec700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75080a2d30 0x7f75080a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f751c1033a0 tx=0x7f750c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.750+0000 7f7522aee700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] conn(0x7f75080ac510 0x7f75080ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.750+0000 7f75137fe700 1 -- 172.21.15.102:0/69737520 <== osd.9 v2:172.21.15.102:6826/3638070029 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7518013e80 con 0x7f75080ac510 2024-04-24T23:11:44.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 --> [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f751c062d40 con 0x7f75080ac510 2024-04-24T23:11:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f75137fe700 1 -- 172.21.15.102:0/69737520 <== osd.9 v2:172.21.15.102:6826/3638070029 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f751c062d40 con 0x7f75080ac510 2024-04-24T23:11:44.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] conn(0x7f75080ac510 msgr2=0x7f75080ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] conn(0x7f75080ac510 0x7f75080ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75080a2d30 msgr2=0x7f75080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f75080a2d30 0x7f75080a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f751c1033a0 tx=0x7f750c005fd0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f751c100780 msgr2=0x7f751c101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 --2- 172.21.15.102:0/69737520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f751c100780 0x7f751c101960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f751800c970 tx=0x7f751800cc80 comp rx=0 tx=0).stop 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7522aee700 1 -- 172.21.15.102:0/69737520 reap_dead start 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 shutdown_connections 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 >> 172.21.15.102:0/69737520 conn(0x7f751c077730 msgr2=0x7f751c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 shutdown_connections 2024-04-24T23:11:44.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:44.770+0000 7f7524551700 1 -- 172.21.15.102:0/69737520 wait complete. 2024-04-24T23:11:45.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:45 smithi102 ceph-mon[29216]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-24T23:11:45.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:45 smithi161 ceph-mon[32709]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-24T23:11:45.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:45 smithi183 ceph-mon[33137]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-24T23:11:45.829 INFO:teuthology.orchestra.run.smithi102.stdout:292057776161 2024-04-24T23:11:45.830 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-24T23:11:46.264 INFO:teuthology.orchestra.run.smithi102.stdout:313532612634 2024-04-24T23:11:46.265 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-24T23:11:46.724 INFO:teuthology.orchestra.run.smithi102.stdout:107374182462 2024-04-24T23:11:46.724 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T23:11:46.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.921+0000 7f3b0438a700 1 -- 172.21.15.102:0/1459226366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3afc1130d0 msgr2=0x7f3afc1154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.921+0000 7f3b0438a700 1 --2- 172.21.15.102:0/1459226366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3afc1130d0 0x7f3afc1154c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009990 tx=0x7f3af8009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.921+0000 7f3b0438a700 1 -- 172.21.15.102:0/1459226366 shutdown_connections 2024-04-24T23:11:46.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.921+0000 7f3b0438a700 1 --2- 172.21.15.102:0/1459226366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3afc1130d0 0x7f3afc1154c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.921+0000 7f3b0438a700 1 --2- 172.21.15.102:0/1459226366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3afc1003b0 0x7f3afc100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.921+0000 7f3b0438a700 1 --2- 172.21.15.102:0/1459226366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3afc100cd0 0x7f3afc112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.921+0000 7f3b0438a700 1 -- 172.21.15.102:0/1459226366 >> 172.21.15.102:0/1459226366 conn(0x7f3afc077330 msgr2=0x7f3afc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:46.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.921+0000 7f3b0438a700 1 -- 172.21.15.102:0/1459226366 shutdown_connections 2024-04-24T23:11:46.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.921+0000 7f3b0438a700 1 -- 172.21.15.102:0/1459226366 wait complete. 2024-04-24T23:11:46.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.922+0000 7f3b0438a700 1 Processor -- start 2024-04-24T23:11:46.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.922+0000 7f3b0438a700 1 -- start start 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.922+0000 7f3b0438a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3afc1003b0 0x7f3afc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.922+0000 7f3b0438a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3afc100cd0 0x7f3afc19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.922+0000 7f3b0438a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3afc1130d0 0x7f3afc1a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.922+0000 7f3b0438a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc118aa0 con 0x7f3afc1003b0 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.922+0000 7f3b0438a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc118920 con 0x7f3afc100cd0 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.922+0000 7f3b0438a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc118c20 con 0x7f3afc1130d0 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02126700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3afc1003b0 0x7f3afc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b01925700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3afc100cd0 0x7f3afc19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02126700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3afc1003b0 0x7f3afc19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35050/0 (socket says 172.21.15.102:35050) 2024-04-24T23:11:46.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02126700 1 -- 172.21.15.102:0/1987940292 learned_addr learned my addr 172.21.15.102:0/1987940292 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02927700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3afc1130d0 0x7f3afc1a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02126700 1 -- 172.21.15.102:0/1987940292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3afc1130d0 msgr2=0x7f3afc1a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02126700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3afc1130d0 0x7f3afc1a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02126700 1 -- 172.21.15.102:0/1987940292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3afc100cd0 msgr2=0x7f3afc19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02126700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3afc100cd0 0x7f3afc19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02126700 1 -- 172.21.15.102:0/1987940292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc1a3f40 con 0x7f3afc1003b0 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b02927700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3afc1130d0 0x7f3afc1a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.923+0000 7f3b01925700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3afc100cd0 0x7f3afc19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.924+0000 7f3b02126700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3afc1003b0 0x7f3afc19ef30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3af4007f10 tx=0x7f3af400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:46.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.924+0000 7f3af37fe700 1 -- 172.21.15.102:0/1987940292 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af400ddd0 con 0x7f3afc1003b0 2024-04-24T23:11:46.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.924+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3afc1a4230 con 0x7f3afc1003b0 2024-04-24T23:11:46.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.924+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3afc1abab0 con 0x7f3afc1003b0 2024-04-24T23:11:46.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.924+0000 7f3af37fe700 1 -- 172.21.15.102:0/1987940292 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3af4004bb0 con 0x7f3afc1003b0 2024-04-24T23:11:46.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.925+0000 7f3af37fe700 1 -- 172.21.15.102:0/1987940292 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40046e0 con 0x7f3afc1003b0 2024-04-24T23:11:46.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.926+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ae0001030 con 0x7f3afc1003b0 2024-04-24T23:11:46.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.928+0000 7f3af37fe700 1 -- 172.21.15.102:0/1987940292 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3af401c020 con 0x7f3afc1003b0 2024-04-24T23:11:46.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.929+0000 7f3af37fe700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ae80a2d30 0x7f3ae80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:46.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.929+0000 7f3b01925700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ae80a2d30 0x7f3ae80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:46.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.930+0000 7f3af37fe700 1 -- 172.21.15.102:0/1987940292 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3af40cf640 con 0x7f3afc1003b0 2024-04-24T23:11:46.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.930+0000 7f3b01925700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ae80a2d30 0x7f3ae80a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3afc107ab0 tx=0x7f3aec009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:46.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.930+0000 7f3af37fe700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] conn(0x7f3ae80ac510 0x7f3ae80ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:46.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.930+0000 7f3af37fe700 1 -- 172.21.15.102:0/1987940292 --> [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3af400cf20 con 0x7f3ae80ac510 2024-04-24T23:11:46.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.930+0000 7f3af37fe700 1 -- 172.21.15.102:0/1987940292 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f3af40cf9f0 con 0x7f3afc1003b0 2024-04-24T23:11:46.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.930+0000 7f3b02927700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] conn(0x7f3ae80ac510 0x7f3ae80ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:46.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.931+0000 7f3b02927700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] conn(0x7f3ae80ac510 0x7f3ae80ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:46.932 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:46.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.932+0000 7f3af37fe700 1 -- 172.21.15.102:0/1987940292 <== osd.4 v2:172.21.15.161:6808/1666138932 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3af400cf20 con 0x7f3ae80ac510 2024-04-24T23:11:46.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.963+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 --> [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ae0002e60 con 0x7f3ae80ac510 2024-04-24T23:11:46.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.964+0000 7f3af37fe700 1 -- 172.21.15.102:0/1987940292 <== osd.4 v2:172.21.15.161:6808/1666138932 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3ae0002e60 con 0x7f3ae80ac510 2024-04-24T23:11:46.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.965+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 >> [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] conn(0x7f3ae80ac510 msgr2=0x7f3ae80ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.965+0000 7f3b0438a700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] conn(0x7f3ae80ac510 0x7f3ae80ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.965+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ae80a2d30 msgr2=0x7f3ae80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.965+0000 7f3b0438a700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3ae80a2d30 0x7f3ae80a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3afc107ab0 tx=0x7f3aec009190 comp rx=0 tx=0).stop 2024-04-24T23:11:46.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.965+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3afc1003b0 msgr2=0x7f3afc19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.965+0000 7f3b0438a700 1 --2- 172.21.15.102:0/1987940292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3afc1003b0 0x7f3afc19ef30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3af4007f10 tx=0x7f3af400d3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.966+0000 7f3b02927700 1 -- 172.21.15.102:0/1987940292 reap_dead start 2024-04-24T23:11:46.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.966+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 shutdown_connections 2024-04-24T23:11:46.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.966+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 >> 172.21.15.102:0/1987940292 conn(0x7f3afc077330 msgr2=0x7f3afc111970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:46.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.966+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 shutdown_connections 2024-04-24T23:11:46.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:46.966+0000 7f3b0438a700 1 -- 172.21.15.102:0/1987940292 wait complete. 2024-04-24T23:11:46.981 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:46.985 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:46.985 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:46.986 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:46.995 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:46.995 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:47.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:47 smithi102 ceph-mon[29216]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:11:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:47 smithi161 ceph-mon[32709]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:11:47.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:47 smithi183 ceph-mon[33137]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T23:11:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-24T23:11:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-24T23:11:48.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-24T23:11:48.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-24T23:11:48.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-24T23:11:48.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-24T23:11:48.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-04-24T23:11:48.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-24T23:11:48.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:48 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]: dispatch 2024-04-24T23:11:48.634 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-24T23:11:48.634 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-24T23:11:48.634 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-24T23:11:48.634 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-24T23:11:48.634 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-24T23:11:48.634 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-24T23:11:48.634 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-04-24T23:11:48.634 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-24T23:11:48.634 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:48 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]: dispatch 2024-04-24T23:11:48.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-24T23:11:48.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-24T23:11:48.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-24T23:11:48.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-24T23:11:48.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-24T23:11:48.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-24T23:11:48.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-04-24T23:11:48.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-24T23:11:48.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:48 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]: dispatch 2024-04-24T23:11:49.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-24T23:11:49.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-24T23:11:49.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-24T23:11:49.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-24T23:11:49.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-24T23:11:49.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-24T23:11:49.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-24T23:11:49.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-04-24T23:11:49.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-24T23:11:49.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]': finished 2024-04-24T23:11:49.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:49 smithi102 ceph-mon[29216]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:11:49.603 INFO:teuthology.orchestra.run.smithi102.stdout:141733920828 2024-04-24T23:11:49.604 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T23:11:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-24T23:11:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-24T23:11:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-24T23:11:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-24T23:11:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-24T23:11:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-24T23:11:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-24T23:11:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-04-24T23:11:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-24T23:11:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]': finished 2024-04-24T23:11:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:49 smithi161 ceph-mon[32709]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:11:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-24T23:11:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-24T23:11:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-24T23:11:49.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-24T23:11:49.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-24T23:11:49.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-24T23:11:49.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-24T23:11:49.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-04-24T23:11:49.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-24T23:11:49.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 8]}]': finished 2024-04-24T23:11:49.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:49 smithi183 ceph-mon[33137]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:11:50.423 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:50 smithi102 ceph-mon[29216]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:11:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:50 smithi161 ceph-mon[32709]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:11:50.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:50 smithi183 ceph-mon[33137]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:11:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:51 smithi161 ceph-mon[32709]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:51.615 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:51 smithi102 ceph-mon[29216]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:51 smithi183 ceph-mon[33137]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:11:52.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.661+0000 7f5b911cc700 1 -- 172.21.15.102:0/1530910073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c101d70 msgr2=0x7f5b8c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:52.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.661+0000 7f5b911cc700 1 --2- 172.21.15.102:0/1530910073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c101d70 0x7f5b8c105d70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5b740099c0 tx=0x7f5b74009cd0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.661+0000 7f5b911cc700 1 -- 172.21.15.102:0/1530910073 shutdown_connections 2024-04-24T23:11:52.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.661+0000 7f5b911cc700 1 --2- 172.21.15.102:0/1530910073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b8c106450 0x7f5b8c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.661+0000 7f5b911cc700 1 --2- 172.21.15.102:0/1530910073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b8c1013c0 0x7f5b8c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.661+0000 7f5b911cc700 1 --2- 172.21.15.102:0/1530910073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c101d70 0x7f5b8c105d70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.661+0000 7f5b911cc700 1 -- 172.21.15.102:0/1530910073 >> 172.21.15.102:0/1530910073 conn(0x7f5b8c078100 msgr2=0x7f5b8c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:52.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.661+0000 7f5b911cc700 1 -- 172.21.15.102:0/1530910073 shutdown_connections 2024-04-24T23:11:52.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.661+0000 7f5b911cc700 1 -- 172.21.15.102:0/1530910073 wait complete. 2024-04-24T23:11:52.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b911cc700 1 Processor -- start 2024-04-24T23:11:52.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b911cc700 1 -- start start 2024-04-24T23:11:52.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b911cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b8c1013c0 0x7f5b8c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:52.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b911cc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b8c101d70 0x7f5b8c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:52.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b911cc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c106450 0x7f5b8c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:52.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b911cc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c1113c0 con 0x7f5b8c106450 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b911cc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c111240 con 0x7f5b8c1013c0 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b911cc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c111540 con 0x7f5b8c101d70 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8b59e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c106450 0x7f5b8c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8b59e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c106450 0x7f5b8c1978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35080/0 (socket says 172.21.15.102:35080) 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8b59e700 1 -- 172.21.15.102:0/3994171391 learned_addr learned my addr 172.21.15.102:0/3994171391 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8ad9d700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b8c1013c0 0x7f5b8c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8259c700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b8c101d70 0x7f5b8c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8b59e700 1 -- 172.21.15.102:0/3994171391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b8c101d70 msgr2=0x7f5b8c19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8b59e700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b8c101d70 0x7f5b8c19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8b59e700 1 -- 172.21.15.102:0/3994171391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b8c1013c0 msgr2=0x7f5b8c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8b59e700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b8c1013c0 0x7f5b8c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.662+0000 7f5b8b59e700 1 -- 172.21.15.102:0/3994171391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c198090 con 0x7f5b8c106450 2024-04-24T23:11:52.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.663+0000 7f5b8b59e700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c106450 0x7f5b8c1978b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400b970 tx=0x7f5b8400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:52.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.663+0000 7f5b88d99700 1 -- 172.21.15.102:0/3994171391 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8400d860 con 0x7f5b8c106450 2024-04-24T23:11:52.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.663+0000 7f5b88d99700 1 -- 172.21.15.102:0/3994171391 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b8400e430 con 0x7f5b8c106450 2024-04-24T23:11:52.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.663+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c198380 con 0x7f5b8c106450 2024-04-24T23:11:52.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.663+0000 7f5b88d99700 1 -- 172.21.15.102:0/3994171391 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84017660 con 0x7f5b8c106450 2024-04-24T23:11:52.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.663+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c1a44f0 con 0x7f5b8c106450 2024-04-24T23:11:52.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.664+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c102850 con 0x7f5b8c106450 2024-04-24T23:11:52.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.665+0000 7f5b88d99700 1 -- 172.21.15.102:0/3994171391 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5b8400e5e0 con 0x7f5b8c106450 2024-04-24T23:11:52.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.666+0000 7f5b88d99700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5b780a2d30 0x7f5b780a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:52.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.666+0000 7f5b8ad9d700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5b780a2d30 0x7f5b780a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:52.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.666+0000 7f5b88d99700 1 -- 172.21.15.102:0/3994171391 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5b840ced50 con 0x7f5b8c106450 2024-04-24T23:11:52.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.667+0000 7f5b8ad9d700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5b780a2d30 0x7f5b780a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009a10 tx=0x7f5b7c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:52.672 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:52.672 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:52.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.672+0000 7f5b88d99700 1 -- 172.21.15.102:0/3994171391 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5b84013030 con 0x7f5b8c106450 2024-04-24T23:11:52.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.881+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5b8c198d40 con 0x7f5b8c106450 2024-04-24T23:11:52.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.881+0000 7f5b88d99700 1 -- 172.21.15.102:0/3994171391 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b8409bb40 con 0x7f5b8c106450 2024-04-24T23:11:52.882 INFO:teuthology.orchestra.run.smithi102.stdout:77309411399 2024-04-24T23:11:52.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5b780a2d30 msgr2=0x7f5b780a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:52.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5b780a2d30 0x7f5b780a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009a10 tx=0x7f5b7c009190 comp rx=0 tx=0).stop 2024-04-24T23:11:52.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c106450 msgr2=0x7f5b8c1978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:52.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c106450 0x7f5b8c1978b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400b970 tx=0x7f5b8400bc80 comp rx=0 tx=0).stop 2024-04-24T23:11:52.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 shutdown_connections 2024-04-24T23:11:52.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f5b780a2d30 0x7f5b780a51f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b8c106450 0x7f5b8c1978b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b8c1013c0 0x7f5b8c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 --2- 172.21.15.102:0/3994171391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b8c101d70 0x7f5b8c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:52.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.883+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 >> 172.21.15.102:0/3994171391 conn(0x7f5b8c078100 msgr2=0x7f5b8c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:52.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.884+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 shutdown_connections 2024-04-24T23:11:52.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:52.884+0000 7f5b911cc700 1 -- 172.21.15.102:0/3994171391 wait complete. 2024-04-24T23:11:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:53 smithi161 ceph-mon[32709]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-04-24T23:11:53.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:53 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3994171391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:11:53.710 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:53 smithi102 ceph-mon[29216]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-04-24T23:11:53.711 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:53 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3994171391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:11:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:53 smithi183 ceph-mon[33137]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-04-24T23:11:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:53 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3994171391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:11:54.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.558+0000 7f8c7fd20700 1 -- 172.21.15.102:0/1813836346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806a7d0 msgr2=0x7f8c7806ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:54.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.558+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/1813836346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806a7d0 0x7f8c7806ac30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009990 tx=0x7f8c68009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.558+0000 7f8c7fd20700 1 -- 172.21.15.102:0/1813836346 shutdown_connections 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.558+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/1813836346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c7806b170 0x7f8c7810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.558+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/1813836346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c781082a0 0x7f8c78108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.558+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/1813836346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806a7d0 0x7f8c7806ac30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.558+0000 7f8c7fd20700 1 -- 172.21.15.102:0/1813836346 >> 172.21.15.102:0/1813836346 conn(0x7f8c780780a0 msgr2=0x7f8c780784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.558+0000 7f8c7fd20700 1 -- 172.21.15.102:0/1813836346 shutdown_connections 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.558+0000 7f8c7fd20700 1 -- 172.21.15.102:0/1813836346 wait complete. 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7fd20700 1 Processor -- start 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7fd20700 1 -- start start 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7fd20700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c7806a7d0 0x7f8c7819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7fd20700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806b170 0x7f8c7819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:54.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7fd20700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c781082a0 0x7f8c781987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7fd20700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78112430 con 0x7f8c7806b170 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7fd20700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781122b0 con 0x7f8c781082a0 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7fd20700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781125b0 con 0x7f8c7806a7d0 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7d2bb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806b170 0x7f8c7819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7d2bb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806b170 0x7f8c7819ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35100/0 (socket says 172.21.15.102:35100) 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7d2bb700 1 -- 172.21.15.102:0/2302829498 learned_addr learned my addr 172.21.15.102:0/2302829498 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.559+0000 7f8c7dabc700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c7806a7d0 0x7f8c7819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7e2bd700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c781082a0 0x7f8c781987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7d2bb700 1 -- 172.21.15.102:0/2302829498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c7806a7d0 msgr2=0x7f8c7819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7d2bb700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c7806a7d0 0x7f8c7819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7d2bb700 1 -- 172.21.15.102:0/2302829498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c781082a0 msgr2=0x7f8c781987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7d2bb700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c781082a0 0x7f8c781987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7d2bb700 1 -- 172.21.15.102:0/2302829498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c78198fd0 con 0x7f8c7806b170 2024-04-24T23:11:54.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7e2bd700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c781082a0 0x7f8c781987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:54.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7dabc700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c7806a7d0 0x7f8c7819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:54.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7d2bb700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806b170 0x7f8c7819ebb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c68000c00 tx=0x7f8c68003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:54.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c6effd700 1 -- 172.21.15.102:0/2302829498 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68027070 con 0x7f8c7806b170 2024-04-24T23:11:54.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c6effd700 1 -- 172.21.15.102:0/2302829498 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c6800bdc0 con 0x7f8c7806b170 2024-04-24T23:11:54.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c78199260 con 0x7f8c7806b170 2024-04-24T23:11:54.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c6effd700 1 -- 172.21.15.102:0/2302829498 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800f460 con 0x7f8c7806b170 2024-04-24T23:11:54.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.560+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781a54d0 con 0x7f8c7806b170 2024-04-24T23:11:54.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.561+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7810a810 con 0x7f8c7806b170 2024-04-24T23:11:54.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.563+0000 7f8c6effd700 1 -- 172.21.15.102:0/2302829498 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8c6802a070 con 0x7f8c7806b170 2024-04-24T23:11:54.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.563+0000 7f8c6effd700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8c640a2cb0 0x7f8c640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:54.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.564+0000 7f8c7dabc700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8c640a2cb0 0x7f8c640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:54.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.564+0000 7f8c6effd700 1 -- 172.21.15.102:0/2302829498 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f8c680d0100 con 0x7f8c7806b170 2024-04-24T23:11:54.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.564+0000 7f8c7dabc700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8c640a2cb0 0x7f8c640a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c740099d0 tx=0x7f8c74008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:54.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.568+0000 7f8c6effd700 1 -- 172.21.15.102:0/2302829498 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8c68013070 con 0x7f8c7806b170 2024-04-24T23:11:54.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.776+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8c78068f50 con 0x7f8c7806b170 2024-04-24T23:11:54.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.776+0000 7f8c6effd700 1 -- 172.21.15.102:0/2302829498 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8c78068f50 con 0x7f8c7806b170 2024-04-24T23:11:54.778 INFO:teuthology.orchestra.run.smithi102.stdout:313532612636 2024-04-24T23:11:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8c640a2cb0 msgr2=0x7f8c640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8c640a2cb0 0x7f8c640a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c740099d0 tx=0x7f8c74008040 comp rx=0 tx=0).stop 2024-04-24T23:11:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806b170 msgr2=0x7f8c7819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806b170 0x7f8c7819ebb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c68000c00 tx=0x7f8c68003b30 comp rx=0 tx=0).stop 2024-04-24T23:11:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 shutdown_connections 2024-04-24T23:11:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c781082a0 0x7f8c781987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c7806a7d0 0x7f8c7819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f8c640a2cb0 0x7f8c640a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 --2- 172.21.15.102:0/2302829498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c7806b170 0x7f8c7819ebb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.778+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 >> 172.21.15.102:0/2302829498 conn(0x7f8c780780a0 msgr2=0x7f8c780ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.779+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 shutdown_connections 2024-04-24T23:11:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:54.779+0000 7f8c7fd20700 1 -- 172.21.15.102:0/2302829498 wait complete. 2024-04-24T23:11:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:55 smithi161 ceph-mon[32709]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-04-24T23:11:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:55 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2302829498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:11:55.633 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:55 smithi102 ceph-mon[29216]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-04-24T23:11:55.633 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:55 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2302829498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:11:55.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.420+0000 7ff14c3d8700 1 -- 172.21.15.102:0/2292732615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff144102e90 msgr2=0x7ff144106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.420+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/2292732615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff144102e90 0x7ff144106ec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff134009990 tx=0x7ff134009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.420+0000 7ff14c3d8700 1 -- 172.21.15.102:0/2292732615 shutdown_connections 2024-04-24T23:11:55.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.420+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/2292732615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1441075a0 0x7ff14410f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.420+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/2292732615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1441024e0 0x7ff1441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.420+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/2292732615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff144102e90 0x7ff144106ec0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.420+0000 7ff14c3d8700 1 -- 172.21.15.102:0/2292732615 >> 172.21.15.102:0/2292732615 conn(0x7ff1440793e0 msgr2=0x7ff1440797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:55.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.420+0000 7ff14c3d8700 1 -- 172.21.15.102:0/2292732615 shutdown_connections 2024-04-24T23:11:55.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.420+0000 7ff14c3d8700 1 -- 172.21.15.102:0/2292732615 wait complete. 2024-04-24T23:11:55.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14c3d8700 1 Processor -- start 2024-04-24T23:11:55.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14c3d8700 1 -- start start 2024-04-24T23:11:55.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14c3d8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1441024e0 0x7ff144074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14c3d8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff144102e90 0x7ff144074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14c3d8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441075a0 0x7ff144074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14c3d8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff144112410 con 0x7ff1441075a0 2024-04-24T23:11:55.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14c3d8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff144112290 con 0x7ff144102e90 2024-04-24T23:11:55.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14c3d8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff144112590 con 0x7ff1441024e0 2024-04-24T23:11:55.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a975700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441075a0 0x7ff144074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a975700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441075a0 0x7ff144074fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35114/0 (socket says 172.21.15.102:35114) 2024-04-24T23:11:55.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a975700 1 -- 172.21.15.102:0/1198057425 learned_addr learned my addr 172.21.15.102:0/1198057425 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:55.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a174700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1441024e0 0x7ff144074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.656 INFO:teuthology.orchestra.run.smithi102.stdout:292057776163 2024-04-24T23:11:55.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a975700 1 -- 172.21.15.102:0/1198057425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1441024e0 msgr2=0x7ff144074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a975700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1441024e0 0x7ff144074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a975700 1 -- 172.21.15.102:0/1198057425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff144102e90 msgr2=0x7ff144074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a975700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff144102e90 0x7ff144074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a975700 1 -- 172.21.15.102:0/1198057425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1441a6e90 con 0x7ff1441075a0 2024-04-24T23:11:55.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.421+0000 7ff14a975700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441075a0 0x7ff144074fb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff14000d6d0 tx=0x7ff14000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:55.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.422+0000 7ff13b7fe700 1 -- 172.21.15.102:0/1198057425 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140017070 con 0x7ff1441075a0 2024-04-24T23:11:55.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.422+0000 7ff13b7fe700 1 -- 172.21.15.102:0/1198057425 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff140005020 con 0x7ff1441075a0 2024-04-24T23:11:55.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.422+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1441a7180 con 0x7ff1441075a0 2024-04-24T23:11:55.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.422+0000 7ff13b7fe700 1 -- 172.21.15.102:0/1198057425 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14000fab0 con 0x7ff1441075a0 2024-04-24T23:11:55.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.422+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1441a7710 con 0x7ff1441075a0 2024-04-24T23:11:55.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.422+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff144068f20 con 0x7ff1441075a0 2024-04-24T23:11:55.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.424+0000 7ff13b7fe700 1 -- 172.21.15.102:0/1198057425 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff1400051d0 con 0x7ff1441075a0 2024-04-24T23:11:55.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.424+0000 7ff13b7fe700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1300a2d30 0x7ff1300a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.424+0000 7ff14a174700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1300a2d30 0x7ff1300a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.424+0000 7ff13b7fe700 1 -- 172.21.15.102:0/1198057425 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff1400cf370 con 0x7ff1441075a0 2024-04-24T23:11:55.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.425+0000 7ff14a174700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1300a2d30 0x7ff1300a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0060b0 tx=0x7ff13c007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:55.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.428+0000 7ff13b7fe700 1 -- 172.21.15.102:0/1198057425 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff140020020 con 0x7ff1441075a0 2024-04-24T23:11:55.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.631+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff14402dcf0 con 0x7ff1441075a0 2024-04-24T23:11:55.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.631+0000 7ff13b7fe700 1 -- 172.21.15.102:0/1198057425 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff14009c1e0 con 0x7ff1441075a0 2024-04-24T23:11:55.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1300a2d30 msgr2=0x7ff1300a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1300a2d30 0x7ff1300a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0060b0 tx=0x7ff13c007b80 comp rx=0 tx=0).stop 2024-04-24T23:11:55.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441075a0 msgr2=0x7ff144074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441075a0 0x7ff144074fb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff14000d6d0 tx=0x7ff14000da90 comp rx=0 tx=0).stop 2024-04-24T23:11:55.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 shutdown_connections 2024-04-24T23:11:55.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1300a2d30 0x7ff1300a51f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441075a0 0x7ff144074fb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1441024e0 0x7ff144074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 --2- 172.21.15.102:0/1198057425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff144102e90 0x7ff144074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 >> 172.21.15.102:0/1198057425 conn(0x7ff1440793e0 msgr2=0x7ff1441005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:55.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 shutdown_connections 2024-04-24T23:11:55.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:55.634+0000 7ff14c3d8700 1 -- 172.21.15.102:0/1198057425 wait complete. 2024-04-24T23:11:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:55 smithi183 ceph-mon[33137]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-04-24T23:11:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:55 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2302829498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:11:56.061 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612634 got 313532612636 for osd.10 2024-04-24T23:11:56.061 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:56.616 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:56 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1198057425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:11:56.662 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:56 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1198057425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:11:56.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:56 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1198057425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:11:57.497 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:57 smithi102 ceph-mon[29216]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.495+0000 7f94539dd700 1 -- 172.21.15.102:0/4110285128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 msgr2=0x7f944c10f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.495+0000 7f94539dd700 1 --2- 172.21.15.102:0/4110285128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 0x7f944c10f1a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f944400b0d0 tx=0x7f944400b3e0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.495+0000 7f94539dd700 1 -- 172.21.15.102:0/4110285128 shutdown_connections 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.495+0000 7f94539dd700 1 --2- 172.21.15.102:0/4110285128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 0x7f944c10f1a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.495+0000 7f94539dd700 1 --2- 172.21.15.102:0/4110285128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f944c06aba0 0x7f944c06af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.495+0000 7f94539dd700 1 --2- 172.21.15.102:0/4110285128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c06b550 0x7f944c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.495+0000 7f94539dd700 1 -- 172.21.15.102:0/4110285128 >> 172.21.15.102:0/4110285128 conn(0x7f944c077e90 msgr2=0x7f944c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.495+0000 7f94539dd700 1 -- 172.21.15.102:0/4110285128 shutdown_connections 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.495+0000 7f94539dd700 1 -- 172.21.15.102:0/4110285128 wait complete. 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f94539dd700 1 Processor -- start 2024-04-24T23:11:57.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f94539dd700 1 -- start start 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f94539dd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f944c06aba0 0x7f944c101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f94539dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c06b550 0x7f944c101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f94539dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 0x7f944c101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f94539dd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1124c0 con 0x7f944c107600 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f94539dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f944c112340 con 0x7f944c06b550 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f94539dd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f944c112640 con 0x7f944c06aba0 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9451f7a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 0x7f944c101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9451f7a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 0x7f944c101fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60154/0 (socket says 172.21.15.102:60154) 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9451f7a700 1 -- 172.21.15.102:0/309741977 learned_addr learned my addr 172.21.15.102:0/309741977 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9451779700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f944c06aba0 0x7f944c101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9450f78700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c06b550 0x7f944c101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9451f7a700 1 -- 172.21.15.102:0/309741977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f944c06aba0 msgr2=0x7f944c101520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9451f7a700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f944c06aba0 0x7f944c101520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9451f7a700 1 -- 172.21.15.102:0/309741977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c06b550 msgr2=0x7f944c101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:57.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9451f7a700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c06b550 0x7f944c101a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9451f7a700 1 -- 172.21.15.102:0/309741977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c1a6e30 con 0x7f944c107600 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.496+0000 7f9450f78700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c06b550 0x7f944c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.497+0000 7f9451f7a700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 0x7f944c101fa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9444009fd0 tx=0x7f9444007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.497+0000 7f94427fc700 1 -- 172.21.15.102:0/309741977 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444021020 con 0x7f944c107600 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.497+0000 7f94427fc700 1 -- 172.21.15.102:0/309741977 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9444004c50 con 0x7f944c107600 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.497+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c1a70c0 con 0x7f944c107600 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.497+0000 7f94427fc700 1 -- 172.21.15.102:0/309741977 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94440052b0 con 0x7f944c107600 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.497+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c1a7570 con 0x7f944c107600 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.498+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c068f20 con 0x7f944c107600 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.499+0000 7f94427fc700 1 -- 172.21.15.102:0/309741977 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9444024020 con 0x7f944c107600 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.499+0000 7f94427fc700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94380a2cb0 0x7f94380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.500+0000 7f9451779700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94380a2cb0 0x7f94380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:57.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.500+0000 7f94427fc700 1 -- 172.21.15.102:0/309741977 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f944400f020 con 0x7f944c107600 2024-04-24T23:11:57.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.500+0000 7f9451779700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94380a2cb0 0x7f94380a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f943c007be0 tx=0x7f943c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:57.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.504+0000 7f94427fc700 1 -- 172.21.15.102:0/309741977 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9444004e00 con 0x7f944c107600 2024-04-24T23:11:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:57 smithi161 ceph-mon[32709]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-04-24T23:11:57.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:57 smithi183 ceph-mon[33137]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-04-24T23:11:57.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.706+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f944c02dcf0 con 0x7f944c107600 2024-04-24T23:11:57.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.706+0000 7f94427fc700 1 -- 172.21.15.102:0/309741977 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f944409d670 con 0x7f944c107600 2024-04-24T23:11:57.709 INFO:teuthology.orchestra.run.smithi102.stdout:201863462963 2024-04-24T23:11:57.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94380a2cb0 msgr2=0x7f94380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:57.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94380a2cb0 0x7f94380a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f943c007be0 tx=0x7f943c0073d0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 msgr2=0x7f944c101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:57.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 0x7f944c101fa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9444009fd0 tx=0x7f9444007a20 comp rx=0 tx=0).stop 2024-04-24T23:11:57.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 shutdown_connections 2024-04-24T23:11:57.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f94380a2cb0 0x7f94380a5170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944c107600 0x7f944c101fa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f944c06aba0 0x7f944c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 --2- 172.21.15.102:0/309741977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f944c06b550 0x7f944c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 >> 172.21.15.102:0/309741977 conn(0x7f944c077e90 msgr2=0x7f944c0ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:57.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 shutdown_connections 2024-04-24T23:11:57.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:11:57.709+0000 7f94539dd700 1 -- 172.21.15.102:0/309741977 wait complete. 2024-04-24T23:11:57.933 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411399 for osd.2 2024-04-24T23:11:57.933 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:58.562 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:58 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/309741977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:11:58.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:58 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/309741977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:11:58.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:58 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/309741977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:11:59.098 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462963 for osd.6 2024-04-24T23:11:59.098 DEBUG:teuthology.parallel:result is None 2024-04-24T23:11:59.621 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:11:59 smithi102 ceph-mon[29216]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5.8 KiB/s, 0 objects/s recovering 2024-04-24T23:11:59.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:11:59 smithi161 ceph-mon[32709]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5.8 KiB/s, 0 objects/s recovering 2024-04-24T23:11:59.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:11:59 smithi183 ceph-mon[33137]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5.8 KiB/s, 0 objects/s recovering 2024-04-24T23:11:59.856 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:59.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776163 for osd.9 2024-04-24T23:11:59.878 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:00.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.461+0000 7fecd0c7c700 1 -- 172.21.15.102:0/3981428166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc107250 msgr2=0x7feccc107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.461+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/3981428166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc107250 0x7feccc107630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0099c0 tx=0x7fecbc009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.461+0000 7fecd0c7c700 1 -- 172.21.15.102:0/3981428166 shutdown_connections 2024-04-24T23:12:00.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.461+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/3981428166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc101140 0x7feccc10f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.461+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/3981428166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc107250 0x7feccc107630 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.461+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/3981428166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feccc100780 0x7feccc100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.461+0000 7fecd0c7c700 1 -- 172.21.15.102:0/3981428166 >> 172.21.15.102:0/3981428166 conn(0x7feccc077730 msgr2=0x7feccc0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.461+0000 7fecd0c7c700 1 -- 172.21.15.102:0/3981428166 shutdown_connections 2024-04-24T23:12:00.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.461+0000 7fecd0c7c700 1 -- 172.21.15.102:0/3981428166 wait complete. 2024-04-24T23:12:00.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecd0c7c700 1 Processor -- start 2024-04-24T23:12:00.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecd0c7c700 1 -- start start 2024-04-24T23:12:00.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecd0c7c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc100780 0x7feccc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecd0c7c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feccc101140 0x7feccc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecd0c7c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc107250 0x7feccc198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecd0c7c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1124e0 con 0x7feccc100780 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecd0c7c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feccc112360 con 0x7feccc101140 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecd0c7c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feccc112660 con 0x7feccc107250 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecca59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc100780 0x7feccc19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecca59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc100780 0x7feccc19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60166/0 (socket says 172.21.15.102:60166) 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.462+0000 7fecca59c700 1 -- 172.21.15.102:0/2457556565 learned_addr learned my addr 172.21.15.102:0/2457556565 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecc9d9b700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feccc101140 0x7feccc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7feccad9d700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc107250 0x7feccc198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecca59c700 1 -- 172.21.15.102:0/2457556565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc107250 msgr2=0x7feccc198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecca59c700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc107250 0x7feccc198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecca59c700 1 -- 172.21.15.102:0/2457556565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feccc101140 msgr2=0x7feccc19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecca59c700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feccc101140 0x7feccc19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecca59c700 1 -- 172.21.15.102:0/2457556565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc198f60 con 0x7feccc100780 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecca59c700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc100780 0x7feccc19e600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fecbc006010 tx=0x7fecbc009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecbb7fe700 1 -- 172.21.15.102:0/2457556565 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc027070 con 0x7feccc100780 2024-04-24T23:12:00.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecbb7fe700 1 -- 172.21.15.102:0/2457556565 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecbc003f50 con 0x7feccc100780 2024-04-24T23:12:00.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc1991f0 con 0x7feccc100780 2024-04-24T23:12:00.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecbb7fe700 1 -- 172.21.15.102:0/2457556565 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc022cd0 con 0x7feccc100780 2024-04-24T23:12:00.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.463+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1a5460 con 0x7feccc100780 2024-04-24T23:12:00.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.464+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc10a7d0 con 0x7feccc100780 2024-04-24T23:12:00.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.465+0000 7fecbb7fe700 1 -- 172.21.15.102:0/2457556565 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fecbc02a070 con 0x7feccc100780 2024-04-24T23:12:00.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.466+0000 7fecbb7fe700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fecb40a2cb0 0x7fecb40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.466+0000 7fecc9d9b700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fecb40a2cb0 0x7fecb40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.467+0000 7fecbb7fe700 1 -- 172.21.15.102:0/2457556565 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fecbc0d03a0 con 0x7feccc100780 2024-04-24T23:12:00.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.467+0000 7fecc9d9b700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fecb40a2cb0 0x7fecb40a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feccc0ffc60 tx=0x7fecc000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.470+0000 7fecbb7fe700 1 -- 172.21.15.102:0/2457556565 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fecbc031080 con 0x7feccc100780 2024-04-24T23:12:00.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.676+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7feccc199cb0 con 0x7feccc100780 2024-04-24T23:12:00.678 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:00 smithi102 ceph-mon[29216]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.3 KiB/s, 0 objects/s recovering 2024-04-24T23:12:00.679 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:00 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.676+0000 7fecbb7fe700 1 -- 172.21.15.102:0/2457556565 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fecbc013070 con 0x7feccc100780 2024-04-24T23:12:00.679 INFO:teuthology.orchestra.run.smithi102.stdout:257698037801 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fecb40a2cb0 msgr2=0x7fecb40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fecb40a2cb0 0x7fecb40a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feccc0ffc60 tx=0x7fecc000b410 comp rx=0 tx=0).stop 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc100780 msgr2=0x7feccc19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc100780 0x7feccc19e600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fecbc006010 tx=0x7fecbc009f80 comp rx=0 tx=0).stop 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 shutdown_connections 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc107250 0x7feccc198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fecb40a2cb0 0x7fecb40a5170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc100780 0x7feccc19e600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 --2- 172.21.15.102:0/2457556565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feccc101140 0x7feccc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 >> 172.21.15.102:0/2457556565 conn(0x7feccc077730 msgr2=0x7feccc105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 shutdown_connections 2024-04-24T23:12:00.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:00.679+0000 7fecd0c7c700 1 -- 172.21.15.102:0/2457556565 wait complete. 2024-04-24T23:12:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:00 smithi183 ceph-mon[33137]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.3 KiB/s, 0 objects/s recovering 2024-04-24T23:12:00.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:00 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:00.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:00 smithi161 ceph-mon[32709]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.3 KiB/s, 0 objects/s recovering 2024-04-24T23:12:00.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:00 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:01 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2457556565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:12:01.706 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:01 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2457556565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:12:01.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:01 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2457556565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:12:02.647 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037797 got 257698037801 for osd.8 2024-04-24T23:12:02.647 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:03.254 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.254 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.254 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.254 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:02 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.254 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:02 smithi102 ceph-mon[29216]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-04-24T23:12:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:02 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:02 smithi183 ceph-mon[33137]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-04-24T23:12:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:02 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:03.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:02 smithi161 ceph-mon[32709]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-04-24T23:12:03.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/3955691257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac101ce0 msgr2=0x7fe7ac102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/3955691257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac101ce0 0x7fe7ac102160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0099c0 tx=0x7fe79c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/3955691257 shutdown_connections 2024-04-24T23:12:03.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/3955691257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7ac1026a0 0x7fe7ac10e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/3955691257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ac107400 0x7fe7ac1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/3955691257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac101ce0 0x7fe7ac102160 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/3955691257 >> 172.21.15.102:0/3955691257 conn(0x7fe7ac076450 msgr2=0x7fe7ac076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/3955691257 shutdown_connections 2024-04-24T23:12:03.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/3955691257 wait complete. 2024-04-24T23:12:03.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.670+0000 7fe7b1c0f700 1 Processor -- start 2024-04-24T23:12:03.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7b1c0f700 1 -- start start 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7b1c0f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7ac101ce0 0x7fe7ac19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7b1c0f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac1026a0 0x7fe7ac19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7b1c0f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ac107400 0x7fe7ac198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7b1c0f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac111420 con 0x7fe7ac1026a0 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7b1c0f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1112a0 con 0x7fe7ac107400 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7b1c0f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1115a0 con 0x7fe7ac101ce0 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7aaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac1026a0 0x7fe7ac19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7aaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac1026a0 0x7fe7ac19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60182/0 (socket says 172.21.15.102:60182) 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7aaffd700 1 -- 172.21.15.102:0/1123317125 learned_addr learned my addr 172.21.15.102:0/1123317125 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7ab7fe700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7ac101ce0 0x7fe7ac19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7aaffd700 1 -- 172.21.15.102:0/1123317125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7ac101ce0 msgr2=0x7fe7ac19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7aaffd700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7ac101ce0 0x7fe7ac19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7aaffd700 1 -- 172.21.15.102:0/1123317125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ac107400 msgr2=0x7fe7ac198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7aaffd700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ac107400 0x7fe7ac198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7aaffd700 1 -- 172.21.15.102:0/1123317125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac199020 con 0x7fe7ac1026a0 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7ab7fe700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7ac101ce0 0x7fe7ac19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.671+0000 7fe7aaffd700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac1026a0 0x7fe7ac19ebb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe79c006010 tx=0x7fe79c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.672+0000 7fe7a8ff9700 1 -- 172.21.15.102:0/1123317125 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c027070 con 0x7fe7ac1026a0 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.672+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac1992b0 con 0x7fe7ac1026a0 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.672+0000 7fe7a8ff9700 1 -- 172.21.15.102:0/1123317125 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe79c003fd0 con 0x7fe7ac1026a0 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.672+0000 7fe7a8ff9700 1 -- 172.21.15.102:0/1123317125 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c022d70 con 0x7fe7ac1026a0 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.672+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac1a5420 con 0x7fe7ac1026a0 2024-04-24T23:12:03.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.673+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac06a580 con 0x7fe7ac1026a0 2024-04-24T23:12:03.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.674+0000 7fe7a8ff9700 1 -- 172.21.15.102:0/1123317125 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe79c02a070 con 0x7fe7ac1026a0 2024-04-24T23:12:03.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.674+0000 7fe7a8ff9700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe7940a2cb0 0x7fe7940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.674+0000 7fe7ab7fe700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe7940a2cb0 0x7fe7940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.674+0000 7fe7a8ff9700 1 -- 172.21.15.102:0/1123317125 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe79c0d02b0 con 0x7fe7ac1026a0 2024-04-24T23:12:03.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.675+0000 7fe7ab7fe700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe7940a2cb0 0x7fe7940a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0005fd0 tx=0x7fe7a0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.678+0000 7fe7a8ff9700 1 -- 172.21.15.102:0/1123317125 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe79c031080 con 0x7fe7ac1026a0 2024-04-24T23:12:03.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.882+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe7ac02db30 con 0x7fe7ac1026a0 2024-04-24T23:12:03.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.882+0000 7fe7a8ff9700 1 -- 172.21.15.102:0/1123317125 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe79c013070 con 0x7fe7ac1026a0 2024-04-24T23:12:03.884 INFO:teuthology.orchestra.run.smithi102.stdout:223338299439 2024-04-24T23:12:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe7940a2cb0 msgr2=0x7fe7940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe7940a2cb0 0x7fe7940a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0005fd0 tx=0x7fe7a0005e30 comp rx=0 tx=0).stop 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac1026a0 msgr2=0x7fe7ac19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac1026a0 0x7fe7ac19ebb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe79c006010 tx=0x7fe79c009f80 comp rx=0 tx=0).stop 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 shutdown_connections 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ac107400 0x7fe7ac198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7ac101ce0 0x7fe7ac19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fe7940a2cb0 0x7fe7940a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 --2- 172.21.15.102:0/1123317125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7ac1026a0 0x7fe7ac19ebb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 >> 172.21.15.102:0/1123317125 conn(0x7fe7ac076450 msgr2=0x7fe7ac100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 shutdown_connections 2024-04-24T23:12:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:03.884+0000 7fe7b1c0f700 1 -- 172.21.15.102:0/1123317125 wait complete. 2024-04-24T23:12:04.094 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:04.200 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:04 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1123317125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:12:04.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 -- 172.21.15.102:0/332023190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c801064b0 msgr2=0x7f3c8010e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 --2- 172.21.15.102:0/332023190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c801064b0 0x7f3c8010e0c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800b0d0 tx=0x7f3c7800b3e0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 -- 172.21.15.102:0/332023190 shutdown_connections 2024-04-24T23:12:04.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 --2- 172.21.15.102:0/332023190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c801064b0 0x7f3c8010e0c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 --2- 172.21.15.102:0/332023190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c80101b90 0x7f3c80101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 --2- 172.21.15.102:0/332023190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c80102540 0x7f3c80105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 -- 172.21.15.102:0/332023190 >> 172.21.15.102:0/332023190 conn(0x7f3c800777a0 msgr2=0x7f3c800feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 -- 172.21.15.102:0/332023190 shutdown_connections 2024-04-24T23:12:04.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 -- 172.21.15.102:0/332023190 wait complete. 2024-04-24T23:12:04.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.301+0000 7f3c84c49700 1 Processor -- start 2024-04-24T23:12:04.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c84c49700 1 -- start start 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c84c49700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c80101b90 0x7f3c8019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c84c49700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c80102540 0x7f3c8019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c84c49700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c801064b0 0x7f3c80197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c84c49700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3c80111480 con 0x7f3c80102540 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c84c49700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3c80111300 con 0x7f3c80101b90 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c84c49700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c80111600 con 0x7f3c801064b0 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7dd9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c80102540 0x7f3c8019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7dd9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c80102540 0x7f3c8019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60208/0 (socket says 172.21.15.102:60208) 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7dd9b700 1 -- 172.21.15.102:0/3555642008 learned_addr learned my addr 172.21.15.102:0/3555642008 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7e59c700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c80101b90 0x7f3c8019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7ed9d700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c801064b0 0x7f3c80197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7dd9b700 1 -- 172.21.15.102:0/3555642008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c801064b0 msgr2=0x7f3c80197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7dd9b700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c801064b0 0x7f3c80197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7dd9b700 1 -- 172.21.15.102:0/3555642008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c80101b90 msgr2=0x7f3c8019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7dd9b700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c80101b90 0x7f3c8019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7dd9b700 1 -- 172.21.15.102:0/3555642008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c80198000 con 0x7f3c80102540 2024-04-24T23:12:04.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c7dd9b700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c80102540 0x7f3c8019db90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400b500 tx=0x7f3c7400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.302+0000 7f3c6f7fe700 1 -- 172.21.15.102:0/3555642008 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74015020 con 0x7f3c80102540 2024-04-24T23:12:04.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.303+0000 7f3c6f7fe700 1 -- 172.21.15.102:0/3555642008 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c74011ce0 con 0x7f3c80102540 2024-04-24T23:12:04.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.303+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c801982f0 con 0x7f3c80102540 2024-04-24T23:12:04.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.303+0000 7f3c6f7fe700 1 -- 172.21.15.102:0/3555642008 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74010d40 con 0x7f3c80102540 2024-04-24T23:12:04.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.303+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c801a44b0 con 0x7f3c80102540 2024-04-24T23:12:04.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.304+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c80068f20 con 0x7f3c80102540 2024-04-24T23:12:04.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.305+0000 7f3c6f7fe700 1 -- 172.21.15.102:0/3555642008 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3c74018020 con 0x7f3c80102540 2024-04-24T23:12:04.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.306+0000 7f3c6f7fe700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3c680a2cb0 0x7f3c680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.306+0000 7f3c7e59c700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3c680a2cb0 0x7f3c680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.306+0000 7f3c6f7fe700 1 -- 172.21.15.102:0/3555642008 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3c740ced90 con 0x7f3c80102540 2024-04-24T23:12:04.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.306+0000 7f3c7e59c700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3c680a2cb0 0x7f3c680a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009a10 tx=0x7f3c70009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.310+0000 7f3c6f7fe700 1 -- 172.21.15.102:0/3555642008 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3c7401f020 con 0x7f3c80102540 2024-04-24T23:12:04.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:04 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1123317125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:12:04.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:04 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1123317125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:12:04.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.514+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3c80102920 con 0x7f3c80102540 2024-04-24T23:12:04.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.515+0000 7f3c6f7fe700 1 -- 172.21.15.102:0/3555642008 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3c7409bc90 con 0x7f3c80102540 2024-04-24T23:12:04.517 INFO:teuthology.orchestra.run.smithi102.stdout:167503724602 2024-04-24T23:12:04.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.517+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3c680a2cb0 msgr2=0x7f3c680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.517+0000 7f3c84c49700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3c680a2cb0 0x7f3c680a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009a10 tx=0x7f3c70009190 comp rx=0 tx=0).stop 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.517+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c80102540 msgr2=0x7f3c8019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.517+0000 7f3c84c49700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c80102540 0x7f3c8019db90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400b500 tx=0x7f3c7400b810 comp rx=0 tx=0).stop 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.518+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 shutdown_connections 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.518+0000 7f3c84c49700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c801064b0 0x7f3c80197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.518+0000 7f3c84c49700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c80101b90 0x7f3c8019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.518+0000 7f3c84c49700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f3c680a2cb0 0x7f3c680a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.518+0000 7f3c84c49700 1 --2- 172.21.15.102:0/3555642008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c80102540 0x7f3c8019db90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.518+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 >> 172.21.15.102:0/3555642008 conn(0x7f3c800777a0 msgr2=0x7f3c800feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.518+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 shutdown_connections 2024-04-24T23:12:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:04.518+0000 7f3c84c49700 1 -- 172.21.15.102:0/3555642008 wait complete. 2024-04-24T23:12:05.134 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:05 smithi102 ceph-mon[29216]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:12:05.134 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:05 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3555642008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:12:05.168 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299439 for osd.7 2024-04-24T23:12:05.169 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:05 smithi161 ceph-mon[32709]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:12:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:05 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3555642008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:12:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:05 smithi183 ceph-mon[33137]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:12:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:05 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3555642008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:12:05.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.707+0000 7f32ec994700 1 -- 172.21.15.102:0/3786569991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8101d70 msgr2=0x7f32e8105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.707+0000 7f32ec994700 1 --2- 172.21.15.102:0/3786569991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8101d70 0x7f32e8105d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f32d00099c0 tx=0x7f32d0009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.707+0000 7f32ec994700 1 -- 172.21.15.102:0/3786569991 shutdown_connections 2024-04-24T23:12:05.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.707+0000 7f32ec994700 1 --2- 172.21.15.102:0/3786569991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32e8106450 0x7f32e810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.707+0000 7f32ec994700 1 --2- 172.21.15.102:0/3786569991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e81013c0 0x7f32e81017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.707+0000 7f32ec994700 1 --2- 172.21.15.102:0/3786569991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8101d70 0x7f32e8105d70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.707+0000 7f32ec994700 1 -- 172.21.15.102:0/3786569991 >> 172.21.15.102:0/3786569991 conn(0x7f32e8078100 msgr2=0x7f32e8078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:05.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.707+0000 7f32ec994700 1 -- 172.21.15.102:0/3786569991 shutdown_connections 2024-04-24T23:12:05.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.707+0000 7f32ec994700 1 -- 172.21.15.102:0/3786569991 wait complete. 2024-04-24T23:12:05.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32ec994700 1 Processor -- start 2024-04-24T23:12:05.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32ec994700 1 -- start start 2024-04-24T23:12:05.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32ec994700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32e81013c0 0x7f32e819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:05.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32ec994700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8101d70 0x7f32e819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:05.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32ec994700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8106450 0x7f32e8197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:05.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32ec994700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8111340 con 0x7f32e8106450 2024-04-24T23:12:05.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32ec994700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81111c0 con 0x7f32e81013c0 2024-04-24T23:12:05.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32ec994700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81114c0 con 0x7f32e8101d70 2024-04-24T23:12:05.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32e6d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8106450 0x7f32e8197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32e6d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8106450 0x7f32e8197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60224/0 (socket says 172.21.15.102:60224) 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.708+0000 7f32e6d9d700 1 -- 172.21.15.102:0/724261545 learned_addr learned my addr 172.21.15.102:0/724261545 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32e5d9b700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8101d70 0x7f32e819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32e659c700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32e81013c0 0x7f32e819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32e6d9d700 1 -- 172.21.15.102:0/724261545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8101d70 msgr2=0x7f32e819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32e6d9d700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8101d70 0x7f32e819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32e6d9d700 1 -- 172.21.15.102:0/724261545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32e81013c0 msgr2=0x7f32e819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32e6d9d700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32e81013c0 0x7f32e819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32e6d9d700 1 -- 172.21.15.102:0/724261545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e8197dc0 con 0x7f32e8106450 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32e659c700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32e81013c0 0x7f32e819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32e6d9d700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8106450 0x7f32e8197660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f32d800c900 tx=0x7f32d800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32df7fe700 1 -- 172.21.15.102:0/724261545 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8007b00 con 0x7f32e8106450 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e81980b0 con 0x7f32e8106450 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32df7fe700 1 -- 172.21.15.102:0/724261545 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d8007ca0 con 0x7f32e8106450 2024-04-24T23:12:05.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32df7fe700 1 -- 172.21.15.102:0/724261545 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8018660 con 0x7f32e8106450 2024-04-24T23:12:05.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.709+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e81a4320 con 0x7f32e8106450 2024-04-24T23:12:05.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.710+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e8068f20 con 0x7f32e8106450 2024-04-24T23:12:05.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.711+0000 7f32df7fe700 1 -- 172.21.15.102:0/724261545 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f32d800f450 con 0x7f32e8106450 2024-04-24T23:12:05.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.712+0000 7f32df7fe700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f32d40a2cb0 0x7f32d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:05.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.712+0000 7f32e659c700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f32d40a2cb0 0x7f32d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:05.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.712+0000 7f32df7fe700 1 -- 172.21.15.102:0/724261545 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f32d80cf370 con 0x7f32e8106450 2024-04-24T23:12:05.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.712+0000 7f32e659c700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f32d40a2cb0 0x7f32d40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32e0007eb0 tx=0x7f32e00073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:05.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.715+0000 7f32df7fe700 1 -- 172.21.15.102:0/724261545 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f32d8014030 con 0x7f32e8106450 2024-04-24T23:12:05.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.917+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f32e8198b40 con 0x7f32e8106450 2024-04-24T23:12:05.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.918+0000 7f32df7fe700 1 -- 172.21.15.102:0/724261545 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f32d809c100 con 0x7f32e8106450 2024-04-24T23:12:05.920 INFO:teuthology.orchestra.run.smithi102.stdout:107374182467 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f32d40a2cb0 msgr2=0x7f32d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f32d40a2cb0 0x7f32d40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32e0007eb0 tx=0x7f32e00073c0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8106450 msgr2=0x7f32e8197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8106450 0x7f32e8197660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f32d800c900 tx=0x7f32d800ccc0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 shutdown_connections 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f32d40a2cb0 0x7f32d40a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32e8106450 0x7f32e8197660 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32e81013c0 0x7f32e819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 --2- 172.21.15.102:0/724261545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32e8101d70 0x7f32e819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 >> 172.21.15.102:0/724261545 conn(0x7f32e8078100 msgr2=0x7f32e80ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:05.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 shutdown_connections 2024-04-24T23:12:05.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:05.920+0000 7f32ec994700 1 -- 172.21.15.102:0/724261545 wait complete. 2024-04-24T23:12:06.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:06 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/724261545' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:12:06.127 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724597 got 167503724602 for osd.5 2024-04-24T23:12:06.127 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:06.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:06 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/724261545' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:12:06.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:06 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/724261545' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:12:07.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:07 smithi102 ceph-mon[29216]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:07 smithi161 ceph-mon[32709]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:07.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182467 for osd.3 2024-04-24T23:12:07.422 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:07 smithi183 ceph-mon[33137]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:08.291 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:09.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:09 smithi102 ceph-mon[29216]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:09 smithi161 ceph-mon[32709]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:09.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.608+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/4221444219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 msgr2=0x7fdc9810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.608+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/4221444219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 0x7fdc9810dfe0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000b0a0 tx=0x7fdc9000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.608+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/4221444219 shutdown_connections 2024-04-24T23:12:09.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.608+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/4221444219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 0x7fdc9810dfe0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.608+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/4221444219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc981071e0 0x7fdc981075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.608+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/4221444219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98101160 0x7fdc981015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.608+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/4221444219 >> 172.21.15.102:0/4221444219 conn(0x7fdc980771a0 msgr2=0x7fdc980775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:09.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.608+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/4221444219 shutdown_connections 2024-04-24T23:12:09.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.608+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/4221444219 wait complete. 2024-04-24T23:12:09.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9f0cc700 1 Processor -- start 2024-04-24T23:12:09.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9f0cc700 1 -- start start 2024-04-24T23:12:09.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9f0cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98101160 0x7fdc9819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:09.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9f0cc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 0x7fdc9819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:09.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9f0cc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc981071e0 0x7fdc981986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9f0cc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98111330 con 0x7fdc98101b20 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9f0cc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981111b0 con 0x7fdc98101160 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9f0cc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981114b0 con 0x7fdc981071e0 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc97fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 0x7fdc9819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc97fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 0x7fdc9819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59964/0 (socket says 172.21.15.102:59964) 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc97fff700 1 -- 172.21.15.102:0/1979822046 learned_addr learned my addr 172.21.15.102:0/1979822046 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9ce68700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98101160 0x7fdc9819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9d669700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc981071e0 0x7fdc981986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc97fff700 1 -- 172.21.15.102:0/1979822046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc981071e0 msgr2=0x7fdc981986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc97fff700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc981071e0 0x7fdc981986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc97fff700 1 -- 172.21.15.102:0/1979822046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98101160 msgr2=0x7fdc9819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc97fff700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98101160 0x7fdc9819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc97fff700 1 -- 172.21.15.102:0/1979822046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc98198f90 con 0x7fdc98101b20 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9d669700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc981071e0 0x7fdc981986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc9ce68700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98101160 0x7fdc9819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.609+0000 7fdc97fff700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 0x7fdc9819eb20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00c9b0 tx=0x7fdc8c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.610+0000 7fdc95ffb700 1 -- 172.21.15.102:0/1979822046 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c007b00 con 0x7fdc98101b20 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.610+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc98199280 con 0x7fdc98101b20 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.610+0000 7fdc95ffb700 1 -- 172.21.15.102:0/1979822046 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc8c007ca0 con 0x7fdc98101b20 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.610+0000 7fdc95ffb700 1 -- 172.21.15.102:0/1979822046 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c018730 con 0x7fdc98101b20 2024-04-24T23:12:09.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.610+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc981a5560 con 0x7fdc98101b20 2024-04-24T23:12:09.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.611+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc98102850 con 0x7fdc98101b20 2024-04-24T23:12:09.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.612+0000 7fdc95ffb700 1 -- 172.21.15.102:0/1979822046 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdc8c00f450 con 0x7fdc98101b20 2024-04-24T23:12:09.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.613+0000 7fdc95ffb700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdc800a2cb0 0x7fdc800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:09.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.613+0000 7fdc9ce68700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdc800a2cb0 0x7fdc800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:09.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.613+0000 7fdc95ffb700 1 -- 172.21.15.102:0/1979822046 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fdc8c0cf2a0 con 0x7fdc98101b20 2024-04-24T23:12:09.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.613+0000 7fdc9ce68700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdc800a2cb0 0x7fdc800a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc880099d0 tx=0x7fdc88008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:09.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.618+0000 7fdc95ffb700 1 -- 172.21.15.102:0/1979822046 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdc8c014030 con 0x7fdc98101b20 2024-04-24T23:12:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:09 smithi183 ceph-mon[33137]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:09.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.824+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fdc98068f20 con 0x7fdc98101b20 2024-04-24T23:12:09.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.824+0000 7fdc95ffb700 1 -- 172.21.15.102:0/1979822046 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fdc8c09c090 con 0x7fdc98101b20 2024-04-24T23:12:09.826 INFO:teuthology.orchestra.run.smithi102.stdout:347892351004 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdc800a2cb0 msgr2=0x7fdc800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdc800a2cb0 0x7fdc800a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc880099d0 tx=0x7fdc88008040 comp rx=0 tx=0).stop 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 msgr2=0x7fdc9819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 0x7fdc9819eb20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00c9b0 tx=0x7fdc8c00cd70 comp rx=0 tx=0).stop 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 shutdown_connections 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc981071e0 0x7fdc981986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98101160 0x7fdc9819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fdc800a2cb0 0x7fdc800a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 --2- 172.21.15.102:0/1979822046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc98101b20 0x7fdc9819eb20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 >> 172.21.15.102:0/1979822046 conn(0x7fdc980771a0 msgr2=0x7fdc981053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 shutdown_connections 2024-04-24T23:12:09.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:09.827+0000 7fdc9f0cc700 1 -- 172.21.15.102:0/1979822046 wait complete. 2024-04-24T23:12:10.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.343+0000 7f762d5bf700 1 -- 172.21.15.102:0/1925313535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 msgr2=0x7f762806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.343+0000 7f762d5bf700 1 --2- 172.21.15.102:0/1925313535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 0x7f762806b020 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f76180099c0 tx=0x7f7618009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.343+0000 7f762d5bf700 1 -- 172.21.15.102:0/1925313535 shutdown_connections 2024-04-24T23:12:10.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.343+0000 7f762d5bf700 1 --2- 172.21.15.102:0/1925313535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7628107590 0x7f762810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.344+0000 7f762d5bf700 1 --2- 172.21.15.102:0/1925313535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 0x7f762806b020 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.344+0000 7f762d5bf700 1 --2- 172.21.15.102:0/1925313535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f762806b5f0 0x7f7628106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.344+0000 7f762d5bf700 1 -- 172.21.15.102:0/1925313535 >> 172.21.15.102:0/1925313535 conn(0x7f76280781d0 msgr2=0x7f76280785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:10.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.344+0000 7f762d5bf700 1 -- 172.21.15.102:0/1925313535 shutdown_connections 2024-04-24T23:12:10.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.344+0000 7f762d5bf700 1 -- 172.21.15.102:0/1925313535 wait complete. 2024-04-24T23:12:10.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.344+0000 7f762d5bf700 1 Processor -- start 2024-04-24T23:12:10.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.344+0000 7f762d5bf700 1 -- start start 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f762d5bf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 0x7f76281011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f762d5bf700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f762806b5f0 0x7f76281016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f762d5bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628107590 0x7f7628101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f762d5bf700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f76281124c0 con 0x7f762806ac40 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f762d5bf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7628112340 con 0x7f7628107590 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f762d5bf700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7628112640 con 0x7f762806b5f0 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 0x7f76281011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 0x7f76281011a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59978/0 (socket says 172.21.15.102:59978) 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f7626ffd700 1 -- 172.21.15.102:0/3545272445 learned_addr learned my addr 172.21.15.102:0/3545272445 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f76267fc700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f762806b5f0 0x7f76281016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f76277fe700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628107590 0x7f7628101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f7626ffd700 1 -- 172.21.15.102:0/3545272445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f762806b5f0 msgr2=0x7f76281016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f7626ffd700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f762806b5f0 0x7f76281016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f7626ffd700 1 -- 172.21.15.102:0/3545272445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628107590 msgr2=0x7f7628101c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f7626ffd700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628107590 0x7f7628101c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f7626ffd700 1 -- 172.21.15.102:0/3545272445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76281a6f00 con 0x7f762806ac40 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f76277fe700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628107590 0x7f7628101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f7626ffd700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 0x7f76281011a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7618006010 tx=0x7f7618009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.345+0000 7f760ffff700 1 -- 172.21.15.102:0/3545272445 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618027070 con 0x7f762806ac40 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.346+0000 7f760ffff700 1 -- 172.21.15.102:0/3545272445 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7618003f50 con 0x7f762806ac40 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.346+0000 7f760ffff700 1 -- 172.21.15.102:0/3545272445 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618022cd0 con 0x7f762806ac40 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.346+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76281a7190 con 0x7f762806ac40 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.346+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76281a7590 con 0x7f762806ac40 2024-04-24T23:12:10.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.346+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7628068f20 con 0x7f762806ac40 2024-04-24T23:12:10.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.348+0000 7f760ffff700 1 -- 172.21.15.102:0/3545272445 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f761802a070 con 0x7f762806ac40 2024-04-24T23:12:10.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.348+0000 7f760ffff700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f76100a2cb0 0x7f76100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:10.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.348+0000 7f76267fc700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f76100a2cb0 0x7f76100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:10.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.348+0000 7f760ffff700 1 -- 172.21.15.102:0/3545272445 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f76180d0340 con 0x7f762806ac40 2024-04-24T23:12:10.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.349+0000 7f76267fc700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f76100a2cb0 0x7f76100a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7628100cc0 tx=0x7f761c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:10.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.352+0000 7f760ffff700 1 -- 172.21.15.102:0/3545272445 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7618031080 con 0x7f762806ac40 2024-04-24T23:12:10.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.551+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f762802dd90 con 0x7f762806ac40 2024-04-24T23:12:10.553 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:10 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1979822046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:12:10.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.551+0000 7f760ffff700 1 -- 172.21.15.102:0/3545272445 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7618013070 con 0x7f762806ac40 2024-04-24T23:12:10.554 INFO:teuthology.orchestra.run.smithi102.stdout:55834574927 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.553+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f76100a2cb0 msgr2=0x7f76100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.553+0000 7f762d5bf700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f76100a2cb0 0x7f76100a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7628100cc0 tx=0x7f761c009190 comp rx=0 tx=0).stop 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 msgr2=0x7f76281011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 0x7f76281011a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7618006010 tx=0x7f7618009f80 comp rx=0 tx=0).stop 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 shutdown_connections 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628107590 0x7f7628101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f76100a2cb0 0x7f76100a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f762806ac40 0x7f76281011a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 --2- 172.21.15.102:0/3545272445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f762806b5f0 0x7f76281016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:10.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 >> 172.21.15.102:0/3545272445 conn(0x7f76280781d0 msgr2=0x7f76280ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:10.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 shutdown_connections 2024-04-24T23:12:10.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:10.554+0000 7f762d5bf700 1 -- 172.21.15.102:0/3545272445 wait complete. 2024-04-24T23:12:10.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:10 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1979822046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:12:10.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:10 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1979822046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:12:11.201 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574927 for osd.1 2024-04-24T23:12:11.201 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:11.490 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:11 smithi102 ceph-mon[29216]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:11.490 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:11 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3545272445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:12:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:11 smithi161 ceph-mon[32709]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:11 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3545272445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:12:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.717+0000 7f55e5a72700 1 -- 172.21.15.102:0/4118944667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e006a8e0 msgr2=0x7f55e010f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.717+0000 7f55e5a72700 1 --2- 172.21.15.102:0/4118944667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e006a8e0 0x7f55e010f0f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009990 tx=0x7f55d0009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.717+0000 7f55e5a72700 1 -- 172.21.15.102:0/4118944667 shutdown_connections 2024-04-24T23:12:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.717+0000 7f55e5a72700 1 --2- 172.21.15.102:0/4118944667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e006a8e0 0x7f55e010f0f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.717+0000 7f55e5a72700 1 --2- 172.21.15.102:0/4118944667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55e01082b0 0x7f55e0108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.717+0000 7f55e5a72700 1 --2- 172.21.15.102:0/4118944667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55e0069f40 0x7f55e006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.717+0000 7f55e5a72700 1 -- 172.21.15.102:0/4118944667 >> 172.21.15.102:0/4118944667 conn(0x7f55e00780e0 msgr2=0x7f55e00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.717+0000 7f55e5a72700 1 -- 172.21.15.102:0/4118944667 shutdown_connections 2024-04-24T23:12:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.717+0000 7f55e5a72700 1 -- 172.21.15.102:0/4118944667 wait complete. 2024-04-24T23:12:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55e5a72700 1 Processor -- start 2024-04-24T23:12:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55e5a72700 1 -- start start 2024-04-24T23:12:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55e5a72700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55e0069f40 0x7f55e019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55e5a72700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55e006a8e0 0x7f55e019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55e5a72700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e01082b0 0x7f55e0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55e5a72700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01123b0 con 0x7f55e01082b0 2024-04-24T23:12:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55e5a72700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0112230 con 0x7f55e006a8e0 2024-04-24T23:12:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55e5a72700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0112530 con 0x7f55e0069f40 2024-04-24T23:12:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55dffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e01082b0 0x7f55e0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55dffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e01082b0 0x7f55e0198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60010/0 (socket says 172.21.15.102:60010) 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55dffff700 1 -- 172.21.15.102:0/310923379 learned_addr learned my addr 172.21.15.102:0/310923379 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55df7fe700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55e0069f40 0x7f55e019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55deffd700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55e006a8e0 0x7f55e019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55dffff700 1 -- 172.21.15.102:0/310923379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55e0069f40 msgr2=0x7f55e019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55dffff700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55e0069f40 0x7f55e019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55dffff700 1 -- 172.21.15.102:0/310923379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55e006a8e0 msgr2=0x7f55e019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55dffff700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55e006a8e0 0x7f55e019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.718+0000 7f55dffff700 1 -- 172.21.15.102:0/310923379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e0198f10 con 0x7f55e01082b0 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.719+0000 7f55deffd700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55e006a8e0 0x7f55e019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.719+0000 7f55dffff700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e01082b0 0x7f55e0198630 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f55d00047b0 tx=0x7f55d0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.719+0000 7f55dcff9700 1 -- 172.21.15.102:0/310923379 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0027070 con 0x7f55e01082b0 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.719+0000 7f55dcff9700 1 -- 172.21.15.102:0/310923379 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55d00040f0 con 0x7f55e01082b0 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.719+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e01991a0 con 0x7f55e01082b0 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.719+0000 7f55dcff9700 1 -- 172.21.15.102:0/310923379 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d000f460 con 0x7f55e01082b0 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.719+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e01a5310 con 0x7f55e01082b0 2024-04-24T23:12:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.720+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e0103960 con 0x7f55e01082b0 2024-04-24T23:12:11.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.721+0000 7f55dcff9700 1 -- 172.21.15.102:0/310923379 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f55d002a070 con 0x7f55e01082b0 2024-04-24T23:12:11.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.721+0000 7f55dcff9700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f55cc0a2cb0 0x7f55cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:11.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.722+0000 7f55df7fe700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f55cc0a2cb0 0x7f55cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:11.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.722+0000 7f55dcff9700 1 -- 172.21.15.102:0/310923379 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f55d00d01a0 con 0x7f55e01082b0 2024-04-24T23:12:11.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.722+0000 7f55df7fe700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f55cc0a2cb0 0x7f55cc0a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f55d8007700 tx=0x7f55d8007690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.726+0000 7f55dcff9700 1 -- 172.21.15.102:0/310923379 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f55d005e740 con 0x7f55e01082b0 2024-04-24T23:12:11.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:11 smithi183 ceph-mon[33137]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:11.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:11 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3545272445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:12:11.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.931+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f55e002db30 con 0x7f55e01082b0 2024-04-24T23:12:11.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.931+0000 7f55dcff9700 1 -- 172.21.15.102:0/310923379 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55d0013070 con 0x7f55e01082b0 2024-04-24T23:12:11.934 INFO:teuthology.orchestra.run.smithi102.stdout:34359738452 2024-04-24T23:12:11.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f55cc0a2cb0 msgr2=0x7f55cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:11.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f55cc0a2cb0 0x7f55cc0a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f55d8007700 tx=0x7f55d8007690 comp rx=0 tx=0).stop 2024-04-24T23:12:11.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e01082b0 msgr2=0x7f55e0198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:11.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e01082b0 0x7f55e0198630 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f55d00047b0 tx=0x7f55d0003730 comp rx=0 tx=0).stop 2024-04-24T23:12:11.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 shutdown_connections 2024-04-24T23:12:11.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f55cc0a2cb0 0x7f55cc0a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55e01082b0 0x7f55e0198630 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55e0069f40 0x7f55e019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 --2- 172.21.15.102:0/310923379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55e006a8e0 0x7f55e019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:11.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 >> 172.21.15.102:0/310923379 conn(0x7f55e00780e0 msgr2=0x7f55e00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:11.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 shutdown_connections 2024-04-24T23:12:11.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:11.934+0000 7f55e5a72700 1 -- 172.21.15.102:0/310923379 wait complete. 2024-04-24T23:12:12.181 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350998 got 347892351004 for osd.11 2024-04-24T23:12:12.181 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:12.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:12 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/310923379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:12:12.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:12 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/310923379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:12:12.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:12 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/310923379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:12:12.812 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738452 for osd.0 2024-04-24T23:12:12.812 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:13.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:13 smithi102 ceph-mon[29216]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-24T23:12:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:13 smithi161 ceph-mon[32709]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-24T23:12:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:13 smithi183 ceph-mon[33137]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-24T23:12:14.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 -- 172.21.15.102:0/84352501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e41071a0 msgr2=0x7f28e4107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 --2- 172.21.15.102:0/84352501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e41071a0 0x7f28e4107580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009990 tx=0x7f28d4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 -- 172.21.15.102:0/84352501 shutdown_connections 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 --2- 172.21.15.102:0/84352501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28e4101af0 0x7f28e410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 --2- 172.21.15.102:0/84352501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e41071a0 0x7f28e4107580 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 --2- 172.21.15.102:0/84352501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e4101130 0x7f28e41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 -- 172.21.15.102:0/84352501 >> 172.21.15.102:0/84352501 conn(0x7f28e40780e0 msgr2=0x7f28e40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 -- 172.21.15.102:0/84352501 shutdown_connections 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 -- 172.21.15.102:0/84352501 wait complete. 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 Processor -- start 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.161+0000 7f28ea843700 1 -- start start 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28ea843700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e4101130 0x7f28e4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28ea843700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e4101af0 0x7f28e4199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28ea843700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28e41071a0 0x7f28e41a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:14.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28ea843700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f28e4111340 con 0x7f28e4101130 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28ea843700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f28e41111c0 con 0x7f28e4101af0 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28ea843700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28e41114c0 con 0x7f28e41071a0 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e4101130 0x7f28e4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e4101130 0x7f28e4199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60034/0 (socket says 172.21.15.102:60034) 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e3fff700 1 -- 172.21.15.102:0/1215112605 learned_addr learned my addr 172.21.15.102:0/1215112605 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e37fe700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e4101af0 0x7f28e4199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e8de0700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28e41071a0 0x7f28e41a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e3fff700 1 -- 172.21.15.102:0/1215112605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28e41071a0 msgr2=0x7f28e41a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e3fff700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28e41071a0 0x7f28e41a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e3fff700 1 -- 172.21.15.102:0/1215112605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e4101af0 msgr2=0x7f28e4199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e3fff700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e4101af0 0x7f28e4199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e3fff700 1 -- 172.21.15.102:0/1215112605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e41a4dc0 con 0x7f28e4101130 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e8de0700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28e41071a0 0x7f28e41a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:14.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e3fff700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e4101130 0x7f28e4199800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f28d4003ab0 tx=0x7f28d4003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e17fa700 1 -- 172.21.15.102:0/1215112605 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4027070 con 0x7f28e4101130 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e41a4f90 con 0x7f28e4101130 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e17fa700 1 -- 172.21.15.102:0/1215112605 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28d400bdc0 con 0x7f28e4101130 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28e17fa700 1 -- 172.21.15.102:0/1215112605 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4022d60 con 0x7f28e4101130 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.162+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e41a53f0 con 0x7f28e4101130 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.163+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e4068f20 con 0x7f28e4101130 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.164+0000 7f28e17fa700 1 -- 172.21.15.102:0/1215112605 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f28d402a070 con 0x7f28e4101130 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.165+0000 7f28e17fa700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28d00a2cb0 0x7f28d00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.165+0000 7f28e37fe700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28d00a2cb0 0x7f28d00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.165+0000 7f28e17fa700 1 -- 172.21.15.102:0/1215112605 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f28d40cff30 con 0x7f28e4101130 2024-04-24T23:12:14.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.165+0000 7f28e37fe700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28d00a2cb0 0x7f28d00a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f28e4078800 tx=0x7f28cc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:14.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.169+0000 7f28e17fa700 1 -- 172.21.15.102:0/1215112605 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f28d4013070 con 0x7f28e4101130 2024-04-24T23:12:14.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.387+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f28e402db30 con 0x7f28e4101130 2024-04-24T23:12:14.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.387+0000 7f28e17fa700 1 -- 172.21.15.102:0/1215112605 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f28d409cdb0 con 0x7f28e4101130 2024-04-24T23:12:14.390 INFO:teuthology.orchestra.run.smithi102.stdout:141733920833 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28d00a2cb0 msgr2=0x7f28d00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28d00a2cb0 0x7f28d00a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f28e4078800 tx=0x7f28cc009190 comp rx=0 tx=0).stop 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e4101130 msgr2=0x7f28e4199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e4101130 0x7f28e4199800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f28d4003ab0 tx=0x7f28d4003ae0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 shutdown_connections 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28e41071a0 0x7f28e41a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f28d00a2cb0 0x7f28d00a5170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e4101130 0x7f28e4199800 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 --2- 172.21.15.102:0/1215112605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28e4101af0 0x7f28e4199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 >> 172.21.15.102:0/1215112605 conn(0x7f28e40780e0 msgr2=0x7f28e40fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 shutdown_connections 2024-04-24T23:12:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:14.390+0000 7f28ea843700 1 -- 172.21.15.102:0/1215112605 wait complete. 2024-04-24T23:12:15.045 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920833 for osd.4 2024-04-24T23:12:15.045 DEBUG:teuthology.parallel:result is None 2024-04-24T23:12:15.045 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T23:12:15.045 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:12:15.298 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:15.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:15 smithi102 ceph-mon[29216]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:12:15.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:15 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1215112605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:12:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:15 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:15 smithi161 ceph-mon[32709]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:12:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:15 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1215112605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:12:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:15.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:15 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:15 smithi183 ceph-mon[33137]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:12:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:15 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1215112605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:12:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:15 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' 2024-04-24T23:12:16.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.951+0000 7fc339bfb700 1 -- 172.21.15.102:0/1983161679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 msgr2=0x7fc3341015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.951+0000 7fc339bfb700 1 --2- 172.21.15.102:0/1983161679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 0x7fc3341015b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc328009990 tx=0x7fc328009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.951+0000 7fc339bfb700 1 -- 172.21.15.102:0/1983161679 shutdown_connections 2024-04-24T23:12:16.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.951+0000 7fc339bfb700 1 --2- 172.21.15.102:0/1983161679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc334101af0 0x7fc33410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.951+0000 7fc339bfb700 1 --2- 172.21.15.102:0/1983161679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3341071a0 0x7fc334107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.951+0000 7fc339bfb700 1 --2- 172.21.15.102:0/1983161679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 0x7fc3341015b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.951+0000 7fc339bfb700 1 -- 172.21.15.102:0/1983161679 >> 172.21.15.102:0/1983161679 conn(0x7fc3340780e0 msgr2=0x7fc3340784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:16.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.951+0000 7fc339bfb700 1 -- 172.21.15.102:0/1983161679 shutdown_connections 2024-04-24T23:12:16.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.951+0000 7fc339bfb700 1 -- 172.21.15.102:0/1983161679 wait complete. 2024-04-24T23:12:16.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.952+0000 7fc339bfb700 1 Processor -- start 2024-04-24T23:12:16.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.952+0000 7fc339bfb700 1 -- start start 2024-04-24T23:12:16.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc339bfb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 0x7fc33419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc339bfb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc334101af0 0x7fc33419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc339bfb700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3341071a0 0x7fc3341987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc339bfb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc334111340 con 0x7fc334101130 2024-04-24T23:12:16.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc339bfb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341111c0 con 0x7fc334101af0 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc339bfb700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341114c0 con 0x7fc3341071a0 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc3337fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 0x7fc33419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc3337fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 0x7fc33419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60724/0 (socket says 172.21.15.102:60724) 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc3337fe700 1 -- 172.21.15.102:0/4014902874 learned_addr learned my addr 172.21.15.102:0/4014902874 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc332ffd700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc334101af0 0x7fc33419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc3337fe700 1 -- 172.21.15.102:0/4014902874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3341071a0 msgr2=0x7fc3341987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc3337fe700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3341071a0 0x7fc3341987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc3337fe700 1 -- 172.21.15.102:0/4014902874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc334101af0 msgr2=0x7fc33419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc3337fe700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc334101af0 0x7fc33419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc3337fe700 1 -- 172.21.15.102:0/4014902874 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc334198fd0 con 0x7fc334101130 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc332ffd700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc334101af0 0x7fc33419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc3337fe700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 0x7fc33419e670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc32400b530 tx=0x7fc32400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc330ff9700 1 -- 172.21.15.102:0/4014902874 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324015020 con 0x7fc334101130 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3341992c0 con 0x7fc334101130 2024-04-24T23:12:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc330ff9700 1 -- 172.21.15.102:0/4014902874 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc324011ce0 con 0x7fc334101130 2024-04-24T23:12:16.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc330ff9700 1 -- 172.21.15.102:0/4014902874 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324010d60 con 0x7fc334101130 2024-04-24T23:12:16.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.953+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3341a54d0 con 0x7fc334101130 2024-04-24T23:12:16.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.954+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc334102850 con 0x7fc334101130 2024-04-24T23:12:16.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.956+0000 7fc330ff9700 1 -- 172.21.15.102:0/4014902874 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc324018020 con 0x7fc334101130 2024-04-24T23:12:16.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.956+0000 7fc330ff9700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc31c0a2cb0 0x7fc31c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.956+0000 7fc332ffd700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc31c0a2cb0 0x7fc31c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.956+0000 7fc330ff9700 1 -- 172.21.15.102:0/4014902874 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc3240ceea0 con 0x7fc334101130 2024-04-24T23:12:16.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.957+0000 7fc332ffd700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc31c0a2cb0 0x7fc31c0a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc328006010 tx=0x7fc32800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:16.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:16.959+0000 7fc330ff9700 1 -- 172.21.15.102:0/4014902874 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc32401f020 con 0x7fc334101130 2024-04-24T23:12:17.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.154+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc334104f50 con 0x7fc31c0a2cb0 2024-04-24T23:12:17.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.161+0000 7fc330ff9700 1 -- 172.21.15.102:0/4014902874 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258990 (secure 0 0 0) 0x7fc334104f50 con 0x7fc31c0a2cb0 2024-04-24T23:12:17.167 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc31c0a2cb0 msgr2=0x7fc31c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc31c0a2cb0 0x7fc31c0a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc328006010 tx=0x7fc32800b540 comp rx=0 tx=0).stop 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 msgr2=0x7fc33419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 0x7fc33419e670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc32400b530 tx=0x7fc32400b8f0 comp rx=0 tx=0).stop 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 shutdown_connections 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3341071a0 0x7fc3341987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc31c0a2cb0 0x7fc31c0a5170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334101130 0x7fc33419e670 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 --2- 172.21.15.102:0/4014902874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc334101af0 0x7fc33419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 >> 172.21.15.102:0/4014902874 conn(0x7fc3340780e0 msgr2=0x7fc334100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:17.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 shutdown_connections 2024-04-24T23:12:17.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:17.164+0000 7fc339bfb700 1 -- 172.21.15.102:0/4014902874 wait complete. 2024-04-24T23:12:17.170 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-24T23:12:17.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:17 smithi102 ceph-mon[29216]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:17 smithi161 ceph-mon[32709]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:17 smithi183 ceph-mon[33137]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:17.955 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-04-24T23:12:16.235254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653906,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":576,"num_read_kb":408,"num_write":337,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1239648,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356720,"kb_used_data":11660,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585104,"statfs":{"total":1151940427776,"available":1151575146496,"internally_reserved":0,"allocated":11939840,"data_stored":7484358,"data_compressed":190013,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002981"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.330059+0000","last_change":"2024-04-24T23:11:01.217761+0000","last_active":"2024-04-24T23:11:49.330059+0000","last_peered":"2024-04-24T23:11:49.330059+0000","last_clean":"2024-04-24T23:11:49.330059+0000","last_became_active":"2024-04-24T23:11:01.217330+0000","last_became_peered":"2024-04-24T23:11:01.217330+0000","last_unstale":"2024-04-24T23:11:49.330059+0000","last_undegraded":"2024-04-24T23:11:49.330059+0000","last_fullsized":"2024-04-24T23:11:49.330059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:10.690940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328775+0000","last_change":"2024-04-24T23:11:01.206722+0000","last_active":"2024-04-24T23:11:49.328775+0000","last_peered":"2024-04-24T23:11:49.328775+0000","last_clean":"2024-04-24T23:11:49.328775+0000","last_became_active":"2024-04-24T23:11:01.205573+0000","last_became_peered":"2024-04-24T23:11:01.205573+0000","last_unstale":"2024-04-24T23:11:49.328775+0000","last_undegraded":"2024-04-24T23:11:49.328775+0000","last_fullsized":"2024-04-24T23:11:49.328775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:01.757792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328830+0000","last_change":"2024-04-24T23:11:01.213159+0000","last_active":"2024-04-24T23:11:49.328830+0000","last_peered":"2024-04-24T23:11:49.328830+0000","last_clean":"2024-04-24T23:11:49.328830+0000","last_became_active":"2024-04-24T23:11:01.212811+0000","last_became_peered":"2024-04-24T23:11:01.212811+0000","last_unstale":"2024-04-24T23:11:49.328830+0000","last_undegraded":"2024-04-24T23:11:49.328830+0000","last_fullsized":"2024-04-24T23:11:49.328830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:02.053206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.711745+0000","last_change":"2024-04-24T23:11:01.213023+0000","last_active":"2024-04-24T23:11:49.711745+0000","last_peered":"2024-04-24T23:11:49.711745+0000","last_clean":"2024-04-24T23:11:49.711745+0000","last_became_active":"2024-04-24T23:11:01.212689+0000","last_became_peered":"2024-04-24T23:11:01.212689+0000","last_unstale":"2024-04-24T23:11:49.711745+0000","last_undegraded":"2024-04-24T23:11:49.711745+0000","last_fullsized":"2024-04-24T23:11:49.711745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:41.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333026+0000","last_change":"2024-04-24T23:11:01.210845+0000","last_active":"2024-04-24T23:11:49.333026+0000","last_peered":"2024-04-24T23:11:49.333026+0000","last_clean":"2024-04-24T23:11:49.333026+0000","last_became_active":"2024-04-24T23:11:01.210389+0000","last_became_peered":"2024-04-24T23:11:01.210389+0000","last_unstale":"2024-04-24T23:11:49.333026+0000","last_undegraded":"2024-04-24T23:11:49.333026+0000","last_fullsized":"2024-04-24T23:11:49.333026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:03.946294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332353+0000","last_change":"2024-04-24T23:11:01.203082+0000","last_active":"2024-04-24T23:11:49.332353+0000","last_peered":"2024-04-24T23:11:49.332353+0000","last_clean":"2024-04-24T23:11:49.332353+0000","last_became_active":"2024-04-24T23:11:01.202605+0000","last_became_peered":"2024-04-24T23:11:01.202605+0000","last_unstale":"2024-04-24T23:11:49.332353+0000","last_undegraded":"2024-04-24T23:11:49.332353+0000","last_fullsized":"2024-04-24T23:11:49.332353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:29.248361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329375+0000","last_change":"2024-04-24T23:11:01.213002+0000","last_active":"2024-04-24T23:11:49.329375+0000","last_peered":"2024-04-24T23:11:49.329375+0000","last_clean":"2024-04-24T23:11:49.329375+0000","last_became_active":"2024-04-24T23:11:01.212690+0000","last_became_peered":"2024-04-24T23:11:01.212690+0000","last_unstale":"2024-04-24T23:11:49.329375+0000","last_undegraded":"2024-04-24T23:11:49.329375+0000","last_fullsized":"2024-04-24T23:11:49.329375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:06.658975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.336156+0000","last_change":"2024-04-24T23:11:49.335234+0000","last_active":"2024-04-24T23:11:49.336156+0000","last_peered":"2024-04-24T23:11:49.336156+0000","last_clean":"2024-04-24T23:11:49.336156+0000","last_became_active":"2024-04-24T23:11:49.334777+0000","last_became_peered":"2024-04-24T23:11:49.334777+0000","last_unstale":"2024-04-24T23:11:49.336156+0000","last_undegraded":"2024-04-24T23:11:49.336156+0000","last_fullsized":"2024-04-24T23:11:49.336156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:41.254260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335531+0000","last_change":"2024-04-24T23:11:01.217638+0000","last_active":"2024-04-24T23:11:49.335531+0000","last_peered":"2024-04-24T23:11:49.335531+0000","last_clean":"2024-04-24T23:11:49.335531+0000","last_became_active":"2024-04-24T23:11:01.217288+0000","last_became_peered":"2024-04-24T23:11:01.217288+0000","last_unstale":"2024-04-24T23:11:49.335531+0000","last_undegraded":"2024-04-24T23:11:49.335531+0000","last_fullsized":"2024-04-24T23:11:49.335531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:24.128868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333154+0000","last_change":"2024-04-24T23:11:01.203188+0000","last_active":"2024-04-24T23:11:49.333154+0000","last_peered":"2024-04-24T23:11:49.333154+0000","last_clean":"2024-04-24T23:11:49.333154+0000","last_became_active":"2024-04-24T23:11:01.202763+0000","last_became_peered":"2024-04-24T23:11:01.202763+0000","last_unstale":"2024-04-24T23:11:49.333154+0000","last_undegraded":"2024-04-24T23:11:49.333154+0000","last_fullsized":"2024-04-24T23:11:49.333154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:04.414650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329773+0000","last_change":"2024-04-24T23:11:01.220641+0000","last_active":"2024-04-24T23:11:49.329773+0000","last_peered":"2024-04-24T23:11:49.329773+0000","last_clean":"2024-04-24T23:11:49.329773+0000","last_became_active":"2024-04-24T23:11:01.220344+0000","last_became_peered":"2024-04-24T23:11:01.220344+0000","last_unstale":"2024-04-24T23:11:49.329773+0000","last_undegraded":"2024-04-24T23:11:49.329773+0000","last_fullsized":"2024-04-24T23:11:49.329773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:44.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335881+0000","last_change":"2024-04-24T23:11:49.334950+0000","last_active":"2024-04-24T23:11:49.335881+0000","last_peered":"2024-04-24T23:11:49.335881+0000","last_clean":"2024-04-24T23:11:49.335881+0000","last_became_active":"2024-04-24T23:11:49.334585+0000","last_became_peered":"2024-04-24T23:11:49.334585+0000","last_unstale":"2024-04-24T23:11:49.335881+0000","last_undegraded":"2024-04-24T23:11:49.335881+0000","last_fullsized":"2024-04-24T23:11:49.335881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:05.223004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334617+0000","last_change":"2024-04-24T23:11:01.205763+0000","last_active":"2024-04-24T23:11:49.334617+0000","last_peered":"2024-04-24T23:11:49.334617+0000","last_clean":"2024-04-24T23:11:49.334617+0000","last_became_active":"2024-04-24T23:11:01.205402+0000","last_became_peered":"2024-04-24T23:11:01.205402+0000","last_unstale":"2024-04-24T23:11:49.334617+0000","last_undegraded":"2024-04-24T23:11:49.334617+0000","last_fullsized":"2024-04-24T23:11:49.334617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:40.976719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328699+0000","last_change":"2024-04-24T23:11:01.209466+0000","last_active":"2024-04-24T23:11:49.328699+0000","last_peered":"2024-04-24T23:11:49.328699+0000","last_clean":"2024-04-24T23:11:49.328699+0000","last_became_active":"2024-04-24T23:11:01.208946+0000","last_became_peered":"2024-04-24T23:11:01.208946+0000","last_unstale":"2024-04-24T23:11:49.328699+0000","last_undegraded":"2024-04-24T23:11:49.328699+0000","last_fullsized":"2024-04-24T23:11:49.328699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:07.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328609+0000","last_change":"2024-04-24T23:11:01.215170+0000","last_active":"2024-04-24T23:11:49.328609+0000","last_peered":"2024-04-24T23:11:49.328609+0000","last_clean":"2024-04-24T23:11:49.328609+0000","last_became_active":"2024-04-24T23:11:01.214867+0000","last_became_peered":"2024-04-24T23:11:01.214867+0000","last_unstale":"2024-04-24T23:11:49.328609+0000","last_undegraded":"2024-04-24T23:11:49.328609+0000","last_fullsized":"2024-04-24T23:11:49.328609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.830234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335026+0000","last_change":"2024-04-24T23:11:01.216691+0000","last_active":"2024-04-24T23:11:49.335026+0000","last_peered":"2024-04-24T23:11:49.335026+0000","last_clean":"2024-04-24T23:11:49.335026+0000","last_became_active":"2024-04-24T23:11:01.216407+0000","last_became_peered":"2024-04-24T23:11:01.216407+0000","last_unstale":"2024-04-24T23:11:49.335026+0000","last_undegraded":"2024-04-24T23:11:49.335026+0000","last_fullsized":"2024-04-24T23:11:49.335026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:03.284374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.330072+0000","last_change":"2024-04-24T23:11:01.217764+0000","last_active":"2024-04-24T23:11:49.330072+0000","last_peered":"2024-04-24T23:11:49.330072+0000","last_clean":"2024-04-24T23:11:49.330072+0000","last_became_active":"2024-04-24T23:11:01.217424+0000","last_became_peered":"2024-04-24T23:11:01.217424+0000","last_unstale":"2024-04-24T23:11:49.330072+0000","last_undegraded":"2024-04-24T23:11:49.330072+0000","last_fullsized":"2024-04-24T23:11:49.330072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:11.517317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326384+0000","last_change":"2024-04-24T23:11:01.214351+0000","last_active":"2024-04-24T23:11:49.326384+0000","last_peered":"2024-04-24T23:11:49.326384+0000","last_clean":"2024-04-24T23:11:49.326384+0000","last_became_active":"2024-04-24T23:11:01.214034+0000","last_became_peered":"2024-04-24T23:11:01.214034+0000","last_unstale":"2024-04-24T23:11:49.326384+0000","last_undegraded":"2024-04-24T23:11:49.326384+0000","last_fullsized":"2024-04-24T23:11:49.326384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:26.105888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332418+0000","last_change":"2024-04-24T23:11:01.212836+0000","last_active":"2024-04-24T23:11:49.332418+0000","last_peered":"2024-04-24T23:11:49.332418+0000","last_clean":"2024-04-24T23:11:49.332418+0000","last_became_active":"2024-04-24T23:11:01.209223+0000","last_became_peered":"2024-04-24T23:11:01.209223+0000","last_unstale":"2024-04-24T23:11:49.332418+0000","last_undegraded":"2024-04-24T23:11:49.332418+0000","last_fullsized":"2024-04-24T23:11:49.332418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:40.100399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326207+0000","last_change":"2024-04-24T23:11:00.302055+0000","last_active":"2024-04-24T23:11:49.326207+0000","last_peered":"2024-04-24T23:11:49.326207+0000","last_clean":"2024-04-24T23:11:49.326207+0000","last_became_active":"2024-04-24T23:11:00.301726+0000","last_became_peered":"2024-04-24T23:11:00.301726+0000","last_unstale":"2024-04-24T23:11:49.326207+0000","last_undegraded":"2024-04-24T23:11:49.326207+0000","last_fullsized":"2024-04-24T23:11:49.326207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:54.039099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331797+0000","last_change":"2024-04-24T23:11:01.219527+0000","last_active":"2024-04-24T23:11:49.331797+0000","last_peered":"2024-04-24T23:11:49.331797+0000","last_clean":"2024-04-24T23:11:49.331797+0000","last_became_active":"2024-04-24T23:11:01.219196+0000","last_became_peered":"2024-04-24T23:11:01.219196+0000","last_unstale":"2024-04-24T23:11:49.331797+0000","last_undegraded":"2024-04-24T23:11:49.331797+0000","last_fullsized":"2024-04-24T23:11:49.331797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:58.584402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331811+0000","last_change":"2024-04-24T23:11:00.309260+0000","last_active":"2024-04-24T23:11:49.331811+0000","last_peered":"2024-04-24T23:11:49.331811+0000","last_clean":"2024-04-24T23:11:49.331811+0000","last_became_active":"2024-04-24T23:11:00.308969+0000","last_became_peered":"2024-04-24T23:11:00.308969+0000","last_unstale":"2024-04-24T23:11:49.331811+0000","last_undegraded":"2024-04-24T23:11:49.331811+0000","last_fullsized":"2024-04-24T23:11:49.331811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:23.137469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328936+0000","last_change":"2024-04-24T23:11:01.216515+0000","last_active":"2024-04-24T23:11:49.328936+0000","last_peered":"2024-04-24T23:11:49.328936+0000","last_clean":"2024-04-24T23:11:49.328936+0000","last_became_active":"2024-04-24T23:11:01.216168+0000","last_became_peered":"2024-04-24T23:11:01.216168+0000","last_unstale":"2024-04-24T23:11:49.328936+0000","last_undegraded":"2024-04-24T23:11:49.328936+0000","last_fullsized":"2024-04-24T23:11:49.328936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:05.229365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333570+0000","last_change":"2024-04-24T23:11:00.302558+0000","last_active":"2024-04-24T23:11:49.333570+0000","last_peered":"2024-04-24T23:11:49.333570+0000","last_clean":"2024-04-24T23:11:49.333570+0000","last_became_active":"2024-04-24T23:11:00.302162+0000","last_became_peered":"2024-04-24T23:11:00.302162+0000","last_unstale":"2024-04-24T23:11:49.333570+0000","last_undegraded":"2024-04-24T23:11:49.333570+0000","last_fullsized":"2024-04-24T23:11:49.333570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:52.623679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335407+0000","last_change":"2024-04-24T23:11:01.222129+0000","last_active":"2024-04-24T23:11:49.335407+0000","last_peered":"2024-04-24T23:11:49.335407+0000","last_clean":"2024-04-24T23:11:49.335407+0000","last_became_active":"2024-04-24T23:11:01.221795+0000","last_became_peered":"2024-04-24T23:11:01.221795+0000","last_unstale":"2024-04-24T23:11:49.335407+0000","last_undegraded":"2024-04-24T23:11:49.335407+0000","last_fullsized":"2024-04-24T23:11:49.335407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:21.417562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334138+0000","last_change":"2024-04-24T23:11:00.308586+0000","last_active":"2024-04-24T23:11:49.334138+0000","last_peered":"2024-04-24T23:11:49.334138+0000","last_clean":"2024-04-24T23:11:49.334138+0000","last_became_active":"2024-04-24T23:11:00.305054+0000","last_became_peered":"2024-04-24T23:11:00.305054+0000","last_unstale":"2024-04-24T23:11:49.334138+0000","last_undegraded":"2024-04-24T23:11:49.334138+0000","last_fullsized":"2024-04-24T23:11:49.334138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:11.371829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.338667+0000","last_change":"2024-04-24T23:11:49.338667+0000","last_active":"2024-04-24T23:11:49.338667+0000","last_peered":"2024-04-24T23:11:49.338667+0000","last_clean":"2024-04-24T23:11:49.338667+0000","last_became_active":"2024-04-24T23:11:49.338218+0000","last_became_peered":"2024-04-24T23:11:49.338218+0000","last_unstale":"2024-04-24T23:11:49.338667+0000","last_undegraded":"2024-04-24T23:11:49.338667+0000","last_fullsized":"2024-04-24T23:11:49.338667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:42.061058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:09.967146+0000","last_change":"2024-04-24T23:11:00.302389+0000","last_active":"2024-04-24T23:12:09.967146+0000","last_peered":"2024-04-24T23:12:09.967146+0000","last_clean":"2024-04-24T23:12:09.967146+0000","last_became_active":"2024-04-24T23:11:00.302061+0000","last_became_peered":"2024-04-24T23:11:00.302061+0000","last_unstale":"2024-04-24T23:12:09.967146+0000","last_undegraded":"2024-04-24T23:12:09.967146+0000","last_fullsized":"2024-04-24T23:12:09.967146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:28.437391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333185+0000","last_change":"2024-04-24T23:11:01.202389+0000","last_active":"2024-04-24T23:11:49.333185+0000","last_peered":"2024-04-24T23:11:49.333185+0000","last_clean":"2024-04-24T23:11:49.333185+0000","last_became_active":"2024-04-24T23:11:01.201986+0000","last_became_peered":"2024-04-24T23:11:01.201986+0000","last_unstale":"2024-04-24T23:11:49.333185+0000","last_undegraded":"2024-04-24T23:11:49.333185+0000","last_fullsized":"2024-04-24T23:11:49.333185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:25.743284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.715664+0000","last_change":"2024-04-24T23:11:49.715664+0000","last_active":"2024-04-24T23:11:49.715664+0000","last_peered":"2024-04-24T23:11:49.715664+0000","last_clean":"2024-04-24T23:11:49.715664+0000","last_became_active":"2024-04-24T23:11:49.712054+0000","last_became_peered":"2024-04-24T23:11:49.712054+0000","last_unstale":"2024-04-24T23:11:49.715664+0000","last_undegraded":"2024-04-24T23:11:49.715664+0000","last_fullsized":"2024-04-24T23:11:49.715664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:51.167456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.336088+0000","last_change":"2024-04-24T23:11:01.214868+0000","last_active":"2024-04-24T23:11:49.336088+0000","last_peered":"2024-04-24T23:11:49.336088+0000","last_clean":"2024-04-24T23:11:49.336088+0000","last_became_active":"2024-04-24T23:11:01.211366+0000","last_became_peered":"2024-04-24T23:11:01.211366+0000","last_unstale":"2024-04-24T23:11:49.336088+0000","last_undegraded":"2024-04-24T23:11:49.336088+0000","last_fullsized":"2024-04-24T23:11:49.336088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:21.595136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334151+0000","last_change":"2024-04-24T23:11:00.302050+0000","last_active":"2024-04-24T23:11:49.334151+0000","last_peered":"2024-04-24T23:11:49.334151+0000","last_clean":"2024-04-24T23:11:49.334151+0000","last_became_active":"2024-04-24T23:11:00.301740+0000","last_became_peered":"2024-04-24T23:11:00.301740+0000","last_unstale":"2024-04-24T23:11:49.334151+0000","last_undegraded":"2024-04-24T23:11:49.334151+0000","last_fullsized":"2024-04-24T23:11:49.334151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:34.994687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331323+0000","last_change":"2024-04-24T23:11:01.211470+0000","last_active":"2024-04-24T23:11:49.331323+0000","last_peered":"2024-04-24T23:11:49.331323+0000","last_clean":"2024-04-24T23:11:49.331323+0000","last_became_active":"2024-04-24T23:11:01.209482+0000","last_became_peered":"2024-04-24T23:11:01.209482+0000","last_unstale":"2024-04-24T23:11:49.331323+0000","last_undegraded":"2024-04-24T23:11:49.331323+0000","last_fullsized":"2024-04-24T23:11:49.331323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:01.407263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335344+0000","last_change":"2024-04-24T23:11:00.305671+0000","last_active":"2024-04-24T23:11:49.335344+0000","last_peered":"2024-04-24T23:11:49.335344+0000","last_clean":"2024-04-24T23:11:49.335344+0000","last_became_active":"2024-04-24T23:11:00.303016+0000","last_became_peered":"2024-04-24T23:11:00.303016+0000","last_unstale":"2024-04-24T23:11:49.335344+0000","last_undegraded":"2024-04-24T23:11:49.335344+0000","last_fullsized":"2024-04-24T23:11:49.335344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:33.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326156+0000","last_change":"2024-04-24T23:11:01.206269+0000","last_active":"2024-04-24T23:11:49.326156+0000","last_peered":"2024-04-24T23:11:49.326156+0000","last_clean":"2024-04-24T23:11:49.326156+0000","last_became_active":"2024-04-24T23:11:01.205941+0000","last_became_peered":"2024-04-24T23:11:01.205941+0000","last_unstale":"2024-04-24T23:11:49.326156+0000","last_undegraded":"2024-04-24T23:11:49.326156+0000","last_fullsized":"2024-04-24T23:11:49.326156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:28.955247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:13.972352+0000","last_change":"2024-04-24T23:11:00.306071+0000","last_active":"2024-04-24T23:12:13.972352+0000","last_peered":"2024-04-24T23:12:13.972352+0000","last_clean":"2024-04-24T23:12:13.972352+0000","last_became_active":"2024-04-24T23:11:00.305751+0000","last_became_peered":"2024-04-24T23:11:00.305751+0000","last_unstale":"2024-04-24T23:12:13.972352+0000","last_undegraded":"2024-04-24T23:12:13.972352+0000","last_fullsized":"2024-04-24T23:12:13.972352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:30.675323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335735+0000","last_change":"2024-04-24T23:11:01.222118+0000","last_active":"2024-04-24T23:11:49.335735+0000","last_peered":"2024-04-24T23:11:49.335735+0000","last_clean":"2024-04-24T23:11:49.335735+0000","last_became_active":"2024-04-24T23:11:01.221830+0000","last_became_peered":"2024-04-24T23:11:01.221830+0000","last_unstale":"2024-04-24T23:11:49.335735+0000","last_undegraded":"2024-04-24T23:11:49.335735+0000","last_fullsized":"2024-04-24T23:11:49.335735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:18.937230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329800+0000","last_change":"2024-04-24T23:11:00.308662+0000","last_active":"2024-04-24T23:11:49.329800+0000","last_peered":"2024-04-24T23:11:49.329800+0000","last_clean":"2024-04-24T23:11:49.329800+0000","last_became_active":"2024-04-24T23:11:00.308355+0000","last_became_peered":"2024-04-24T23:11:00.308355+0000","last_unstale":"2024-04-24T23:11:49.329800+0000","last_undegraded":"2024-04-24T23:11:49.329800+0000","last_fullsized":"2024-04-24T23:11:49.329800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:15.228277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.338279+0000","last_change":"2024-04-24T23:11:49.338279+0000","last_active":"2024-04-24T23:11:49.338279+0000","last_peered":"2024-04-24T23:11:49.338279+0000","last_clean":"2024-04-24T23:11:49.338279+0000","last_became_active":"2024-04-24T23:11:49.338030+0000","last_became_peered":"2024-04-24T23:11:49.338030+0000","last_unstale":"2024-04-24T23:11:49.338279+0000","last_undegraded":"2024-04-24T23:11:49.338279+0000","last_fullsized":"2024-04-24T23:11:49.338279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:30.867560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.336108+0000","last_change":"2024-04-24T23:11:00.308114+0000","last_active":"2024-04-24T23:11:49.336108+0000","last_peered":"2024-04-24T23:11:49.336108+0000","last_clean":"2024-04-24T23:11:49.336108+0000","last_became_active":"2024-04-24T23:11:00.306632+0000","last_became_peered":"2024-04-24T23:11:00.306632+0000","last_unstale":"2024-04-24T23:11:49.336108+0000","last_undegraded":"2024-04-24T23:11:49.336108+0000","last_fullsized":"2024-04-24T23:11:49.336108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:19.298736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335554+0000","last_change":"2024-04-24T23:11:01.219789+0000","last_active":"2024-04-24T23:11:49.335554+0000","last_peered":"2024-04-24T23:11:49.335554+0000","last_clean":"2024-04-24T23:11:49.335554+0000","last_became_active":"2024-04-24T23:11:01.219498+0000","last_became_peered":"2024-04-24T23:11:01.219498+0000","last_unstale":"2024-04-24T23:11:49.335554+0000","last_undegraded":"2024-04-24T23:11:49.335554+0000","last_fullsized":"2024-04-24T23:11:49.335554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:26.076594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:14.755395+0000","last_change":"2024-04-24T23:11:49.721720+0000","last_active":"2024-04-24T23:12:14.755395+0000","last_peered":"2024-04-24T23:12:14.755395+0000","last_clean":"2024-04-24T23:12:14.755395+0000","last_became_active":"2024-04-24T23:11:49.712796+0000","last_became_peered":"2024-04-24T23:11:49.712796+0000","last_unstale":"2024-04-24T23:12:14.755395+0000","last_undegraded":"2024-04-24T23:12:14.755395+0000","last_fullsized":"2024-04-24T23:12:14.755395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:36.431137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":147,"num_read_kb":130,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":58912,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333824+0000","last_change":"2024-04-24T23:11:01.204997+0000","last_active":"2024-04-24T23:11:49.333824+0000","last_peered":"2024-04-24T23:11:49.333824+0000","last_clean":"2024-04-24T23:11:49.333824+0000","last_became_active":"2024-04-24T23:11:01.204645+0000","last_became_peered":"2024-04-24T23:11:01.204645+0000","last_unstale":"2024-04-24T23:11:49.333824+0000","last_undegraded":"2024-04-24T23:11:49.333824+0000","last_fullsized":"2024-04-24T23:11:49.333824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:07.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332025+0000","last_change":"2024-04-24T23:11:00.302465+0000","last_active":"2024-04-24T23:11:49.332025+0000","last_peered":"2024-04-24T23:11:49.332025+0000","last_clean":"2024-04-24T23:11:49.332025+0000","last_became_active":"2024-04-24T23:11:00.302162+0000","last_became_peered":"2024-04-24T23:11:00.302162+0000","last_unstale":"2024-04-24T23:11:49.332025+0000","last_undegraded":"2024-04-24T23:11:49.332025+0000","last_fullsized":"2024-04-24T23:11:49.332025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:51.352872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334477+0000","last_change":"2024-04-24T23:11:01.206588+0000","last_active":"2024-04-24T23:11:49.334477+0000","last_peered":"2024-04-24T23:11:49.334477+0000","last_clean":"2024-04-24T23:11:49.334477+0000","last_became_active":"2024-04-24T23:11:01.206258+0000","last_became_peered":"2024-04-24T23:11:01.206258+0000","last_unstale":"2024-04-24T23:11:49.334477+0000","last_undegraded":"2024-04-24T23:11:49.334477+0000","last_fullsized":"2024-04-24T23:11:49.334477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:26.857203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335184+0000","last_change":"2024-04-24T23:11:00.305179+0000","last_active":"2024-04-24T23:11:49.335184+0000","last_peered":"2024-04-24T23:11:49.335184+0000","last_clean":"2024-04-24T23:11:49.335184+0000","last_became_active":"2024-04-24T23:11:00.302951+0000","last_became_peered":"2024-04-24T23:11:00.302951+0000","last_unstale":"2024-04-24T23:11:49.335184+0000","last_undegraded":"2024-04-24T23:11:49.335184+0000","last_fullsized":"2024-04-24T23:11:49.335184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:33.278498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329628+0000","last_change":"2024-04-24T23:11:01.211062+0000","last_active":"2024-04-24T23:11:49.329628+0000","last_peered":"2024-04-24T23:11:49.329628+0000","last_clean":"2024-04-24T23:11:49.329628+0000","last_became_active":"2024-04-24T23:11:01.210752+0000","last_became_peered":"2024-04-24T23:11:01.210752+0000","last_unstale":"2024-04-24T23:11:49.329628+0000","last_undegraded":"2024-04-24T23:11:49.329628+0000","last_fullsized":"2024-04-24T23:11:49.329628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:54.790212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:09.978460+0000","last_change":"2024-04-24T23:11:00.305690+0000","last_active":"2024-04-24T23:12:09.978460+0000","last_peered":"2024-04-24T23:12:09.978460+0000","last_clean":"2024-04-24T23:12:09.978460+0000","last_became_active":"2024-04-24T23:11:00.305386+0000","last_became_peered":"2024-04-24T23:11:00.305386+0000","last_unstale":"2024-04-24T23:12:09.978460+0000","last_undegraded":"2024-04-24T23:12:09.978460+0000","last_fullsized":"2024-04-24T23:12:09.978460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:39.898085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329244+0000","last_change":"2024-04-24T23:11:01.208916+0000","last_active":"2024-04-24T23:11:49.329244+0000","last_peered":"2024-04-24T23:11:49.329244+0000","last_clean":"2024-04-24T23:11:49.329244+0000","last_became_active":"2024-04-24T23:11:01.208593+0000","last_became_peered":"2024-04-24T23:11:01.208593+0000","last_unstale":"2024-04-24T23:11:49.329244+0000","last_undegraded":"2024-04-24T23:11:49.329244+0000","last_fullsized":"2024-04-24T23:11:49.329244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:58.442365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332974+0000","last_change":"2024-04-24T23:11:00.309608+0000","last_active":"2024-04-24T23:11:49.332974+0000","last_peered":"2024-04-24T23:11:49.332974+0000","last_clean":"2024-04-24T23:11:49.332974+0000","last_became_active":"2024-04-24T23:11:00.309310+0000","last_became_peered":"2024-04-24T23:11:00.309310+0000","last_unstale":"2024-04-24T23:11:49.332974+0000","last_undegraded":"2024-04-24T23:11:49.332974+0000","last_fullsized":"2024-04-24T23:11:49.332974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:23.330558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335086+0000","last_change":"2024-04-24T23:11:01.212376+0000","last_active":"2024-04-24T23:11:49.335086+0000","last_peered":"2024-04-24T23:11:49.335086+0000","last_clean":"2024-04-24T23:11:49.335086+0000","last_became_active":"2024-04-24T23:11:01.212043+0000","last_became_peered":"2024-04-24T23:11:01.212043+0000","last_unstale":"2024-04-24T23:11:49.335086+0000","last_undegraded":"2024-04-24T23:11:49.335086+0000","last_fullsized":"2024-04-24T23:11:49.335086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:54.025829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332970+0000","last_change":"2024-04-24T23:11:00.305483+0000","last_active":"2024-04-24T23:11:49.332970+0000","last_peered":"2024-04-24T23:11:49.332970+0000","last_clean":"2024-04-24T23:11:49.332970+0000","last_became_active":"2024-04-24T23:11:00.305186+0000","last_became_peered":"2024-04-24T23:11:00.305186+0000","last_unstale":"2024-04-24T23:11:49.332970+0000","last_undegraded":"2024-04-24T23:11:49.332970+0000","last_fullsized":"2024-04-24T23:11:49.332970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:55.633200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329796+0000","last_change":"2024-04-24T23:11:01.210803+0000","last_active":"2024-04-24T23:11:49.329796+0000","last_peered":"2024-04-24T23:11:49.329796+0000","last_clean":"2024-04-24T23:11:49.329796+0000","last_became_active":"2024-04-24T23:11:01.210154+0000","last_became_peered":"2024-04-24T23:11:01.210154+0000","last_unstale":"2024-04-24T23:11:49.329796+0000","last_undegraded":"2024-04-24T23:11:49.329796+0000","last_fullsized":"2024-04-24T23:11:49.329796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:08.106252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326237+0000","last_change":"2024-04-24T23:11:00.305518+0000","last_active":"2024-04-24T23:11:49.326237+0000","last_peered":"2024-04-24T23:11:49.326237+0000","last_clean":"2024-04-24T23:11:49.326237+0000","last_became_active":"2024-04-24T23:11:00.305192+0000","last_became_peered":"2024-04-24T23:11:00.305192+0000","last_unstale":"2024-04-24T23:11:49.326237+0000","last_undegraded":"2024-04-24T23:11:49.326237+0000","last_fullsized":"2024-04-24T23:11:49.326237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:43:36.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332438+0000","last_change":"2024-04-24T23:11:01.207723+0000","last_active":"2024-04-24T23:11:49.332438+0000","last_peered":"2024-04-24T23:11:49.332438+0000","last_clean":"2024-04-24T23:11:49.332438+0000","last_became_active":"2024-04-24T23:11:01.207431+0000","last_became_peered":"2024-04-24T23:11:01.207431+0000","last_unstale":"2024-04-24T23:11:49.332438+0000","last_undegraded":"2024-04-24T23:11:49.332438+0000","last_fullsized":"2024-04-24T23:11:49.332438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:06.883404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329515+0000","last_change":"2024-04-24T23:11:00.310700+0000","last_active":"2024-04-24T23:11:49.329515+0000","last_peered":"2024-04-24T23:11:49.329515+0000","last_clean":"2024-04-24T23:11:49.329515+0000","last_became_active":"2024-04-24T23:11:00.310216+0000","last_became_peered":"2024-04-24T23:11:00.310216+0000","last_unstale":"2024-04-24T23:11:49.329515+0000","last_undegraded":"2024-04-24T23:11:49.329515+0000","last_fullsized":"2024-04-24T23:11:49.329515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:25.326695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332744+0000","last_change":"2024-04-24T23:11:01.214385+0000","last_active":"2024-04-24T23:11:49.332744+0000","last_peered":"2024-04-24T23:11:49.332744+0000","last_clean":"2024-04-24T23:11:49.332744+0000","last_became_active":"2024-04-24T23:11:01.214090+0000","last_became_peered":"2024-04-24T23:11:01.214090+0000","last_unstale":"2024-04-24T23:11:49.332744+0000","last_undegraded":"2024-04-24T23:11:49.332744+0000","last_fullsized":"2024-04-24T23:11:49.332744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:57.024833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.711776+0000","last_change":"2024-04-24T23:11:49.711351+0000","last_active":"2024-04-24T23:11:49.711776+0000","last_peered":"2024-04-24T23:11:49.711776+0000","last_clean":"2024-04-24T23:11:49.711776+0000","last_became_active":"2024-04-24T23:11:49.710985+0000","last_became_peered":"2024-04-24T23:11:49.710985+0000","last_unstale":"2024-04-24T23:11:49.711776+0000","last_undegraded":"2024-04-24T23:11:49.711776+0000","last_fullsized":"2024-04-24T23:11:49.711776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:42.064044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:10.979562+0000","last_change":"2024-04-24T23:11:00.305349+0000","last_active":"2024-04-24T23:12:10.979562+0000","last_peered":"2024-04-24T23:12:10.979562+0000","last_clean":"2024-04-24T23:12:10.979562+0000","last_became_active":"2024-04-24T23:11:00.305046+0000","last_became_peered":"2024-04-24T23:11:00.305046+0000","last_unstale":"2024-04-24T23:12:10.979562+0000","last_undegraded":"2024-04-24T23:12:10.979562+0000","last_fullsized":"2024-04-24T23:12:10.979562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:56.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331654+0000","last_change":"2024-04-24T23:11:01.216209+0000","last_active":"2024-04-24T23:11:49.331654+0000","last_peered":"2024-04-24T23:11:49.331654+0000","last_clean":"2024-04-24T23:11:49.331654+0000","last_became_active":"2024-04-24T23:11:01.215899+0000","last_became_peered":"2024-04-24T23:11:01.215899+0000","last_unstale":"2024-04-24T23:11:49.331654+0000","last_undegraded":"2024-04-24T23:11:49.331654+0000","last_fullsized":"2024-04-24T23:11:49.331654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:09.404193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.711060+0000","last_change":"2024-04-24T23:11:00.303160+0000","last_active":"2024-04-24T23:11:49.711060+0000","last_peered":"2024-04-24T23:11:49.711060+0000","last_clean":"2024-04-24T23:11:49.711060+0000","last_became_active":"2024-04-24T23:11:00.302781+0000","last_became_peered":"2024-04-24T23:11:00.302781+0000","last_unstale":"2024-04-24T23:11:49.711060+0000","last_undegraded":"2024-04-24T23:11:49.711060+0000","last_fullsized":"2024-04-24T23:11:49.711060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:42.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329413+0000","last_change":"2024-04-24T23:09:43.738402+0000","last_active":"2024-04-24T23:11:49.329413+0000","last_peered":"2024-04-24T23:11:49.329413+0000","last_clean":"2024-04-24T23:11:49.329413+0000","last_became_active":"2024-04-24T23:09:43.737979+0000","last_became_peered":"2024-04-24T23:09:43.737979+0000","last_unstale":"2024-04-24T23:11:49.329413+0000","last_undegraded":"2024-04-24T23:11:49.329413+0000","last_fullsized":"2024-04-24T23:11:49.329413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:16.456818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:16.456818+0000","last_clean_scrub_stamp":"2024-04-24T23:06:16.456818+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:52.449055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334022+0000","last_change":"2024-04-24T23:11:01.203542+0000","last_active":"2024-04-24T23:11:49.334022+0000","last_peered":"2024-04-24T23:11:49.334022+0000","last_clean":"2024-04-24T23:11:49.334022+0000","last_became_active":"2024-04-24T23:11:01.203111+0000","last_became_peered":"2024-04-24T23:11:01.203111+0000","last_unstale":"2024-04-24T23:11:49.334022+0000","last_undegraded":"2024-04-24T23:11:49.334022+0000","last_fullsized":"2024-04-24T23:11:49.334022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:27.833165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329511+0000","last_change":"2024-04-24T23:11:00.304542+0000","last_active":"2024-04-24T23:11:49.329511+0000","last_peered":"2024-04-24T23:11:49.329511+0000","last_clean":"2024-04-24T23:11:49.329511+0000","last_became_active":"2024-04-24T23:11:00.304234+0000","last_became_peered":"2024-04-24T23:11:00.304234+0000","last_unstale":"2024-04-24T23:11:49.329511+0000","last_undegraded":"2024-04-24T23:11:49.329511+0000","last_fullsized":"2024-04-24T23:11:49.329511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:18.318876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333377+0000","last_change":"2024-04-24T23:11:00.310962+0000","last_active":"2024-04-24T23:11:49.333377+0000","last_peered":"2024-04-24T23:11:49.333377+0000","last_clean":"2024-04-24T23:11:49.333377+0000","last_became_active":"2024-04-24T23:11:00.310648+0000","last_became_peered":"2024-04-24T23:11:00.310648+0000","last_unstale":"2024-04-24T23:11:49.333377+0000","last_undegraded":"2024-04-24T23:11:49.333377+0000","last_fullsized":"2024-04-24T23:11:49.333377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:01.693517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334835+0000","last_change":"2024-04-24T23:11:01.200293+0000","last_active":"2024-04-24T23:11:49.334835+0000","last_peered":"2024-04-24T23:11:49.334835+0000","last_clean":"2024-04-24T23:11:49.334835+0000","last_became_active":"2024-04-24T23:11:01.199945+0000","last_became_peered":"2024-04-24T23:11:01.199945+0000","last_unstale":"2024-04-24T23:11:49.334835+0000","last_undegraded":"2024-04-24T23:11:49.334835+0000","last_fullsized":"2024-04-24T23:11:49.334835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:25.258253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326436+0000","last_change":"2024-04-24T23:11:00.307467+0000","last_active":"2024-04-24T23:11:49.326436+0000","last_peered":"2024-04-24T23:11:49.326436+0000","last_clean":"2024-04-24T23:11:49.326436+0000","last_became_active":"2024-04-24T23:11:00.307147+0000","last_became_peered":"2024-04-24T23:11:00.307147+0000","last_unstale":"2024-04-24T23:11:49.326436+0000","last_undegraded":"2024-04-24T23:11:49.326436+0000","last_fullsized":"2024-04-24T23:11:49.326436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:42.955410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331886+0000","last_change":"2024-04-24T23:11:01.210541+0000","last_active":"2024-04-24T23:11:49.331886+0000","last_peered":"2024-04-24T23:11:49.331886+0000","last_clean":"2024-04-24T23:11:49.331886+0000","last_became_active":"2024-04-24T23:11:01.210187+0000","last_became_peered":"2024-04-24T23:11:01.210187+0000","last_unstale":"2024-04-24T23:11:49.331886+0000","last_undegraded":"2024-04-24T23:11:49.331886+0000","last_fullsized":"2024-04-24T23:11:49.331886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:45.138001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.711027+0000","last_change":"2024-04-24T23:11:00.308782+0000","last_active":"2024-04-24T23:11:49.711027+0000","last_peered":"2024-04-24T23:11:49.711027+0000","last_clean":"2024-04-24T23:11:49.711027+0000","last_became_active":"2024-04-24T23:11:00.308269+0000","last_became_peered":"2024-04-24T23:11:00.308269+0000","last_unstale":"2024-04-24T23:11:49.711027+0000","last_undegraded":"2024-04-24T23:11:49.711027+0000","last_fullsized":"2024-04-24T23:11:49.711027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:05.089037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335979+0000","last_change":"2024-04-24T23:11:49.335979+0000","last_active":"2024-04-24T23:11:49.335979+0000","last_peered":"2024-04-24T23:11:49.335979+0000","last_clean":"2024-04-24T23:11:49.335979+0000","last_became_active":"2024-04-24T23:11:49.335562+0000","last_became_peered":"2024-04-24T23:11:49.335562+0000","last_unstale":"2024-04-24T23:11:49.335979+0000","last_undegraded":"2024-04-24T23:11:49.335979+0000","last_fullsized":"2024-04-24T23:11:49.335979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:45.810023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335666+0000","last_change":"2024-04-24T23:11:00.307522+0000","last_active":"2024-04-24T23:11:49.335666+0000","last_peered":"2024-04-24T23:11:49.335666+0000","last_clean":"2024-04-24T23:11:49.335666+0000","last_became_active":"2024-04-24T23:11:00.306106+0000","last_became_peered":"2024-04-24T23:11:00.306106+0000","last_unstale":"2024-04-24T23:11:49.335666+0000","last_undegraded":"2024-04-24T23:11:49.335666+0000","last_fullsized":"2024-04-24T23:11:49.335666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:21.623996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332269+0000","last_change":"2024-04-24T23:11:01.207426+0000","last_active":"2024-04-24T23:11:49.332269+0000","last_peered":"2024-04-24T23:11:49.332269+0000","last_clean":"2024-04-24T23:11:49.332269+0000","last_became_active":"2024-04-24T23:11:01.207020+0000","last_became_peered":"2024-04-24T23:11:01.207020+0000","last_unstale":"2024-04-24T23:11:49.332269+0000","last_undegraded":"2024-04-24T23:11:49.332269+0000","last_fullsized":"2024-04-24T23:11:49.332269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:58.389998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331365+0000","last_change":"2024-04-24T23:11:00.302686+0000","last_active":"2024-04-24T23:11:49.331365+0000","last_peered":"2024-04-24T23:11:49.331365+0000","last_clean":"2024-04-24T23:11:49.331365+0000","last_became_active":"2024-04-24T23:11:00.302362+0000","last_became_peered":"2024-04-24T23:11:00.302362+0000","last_unstale":"2024-04-24T23:11:49.331365+0000","last_undegraded":"2024-04-24T23:11:49.331365+0000","last_fullsized":"2024-04-24T23:11:49.331365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:30.402780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329504+0000","last_change":"2024-04-24T23:11:01.212413+0000","last_active":"2024-04-24T23:11:49.329504+0000","last_peered":"2024-04-24T23:11:49.329504+0000","last_clean":"2024-04-24T23:11:49.329504+0000","last_became_active":"2024-04-24T23:11:01.211975+0000","last_became_peered":"2024-04-24T23:11:01.211975+0000","last_unstale":"2024-04-24T23:11:49.329504+0000","last_undegraded":"2024-04-24T23:11:49.329504+0000","last_fullsized":"2024-04-24T23:11:49.329504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:56.129977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.339617+0000","last_change":"2024-04-24T23:11:49.339214+0000","last_active":"2024-04-24T23:11:49.339617+0000","last_peered":"2024-04-24T23:11:49.339617+0000","last_clean":"2024-04-24T23:11:49.339617+0000","last_became_active":"2024-04-24T23:11:49.334513+0000","last_became_peered":"2024-04-24T23:11:49.334513+0000","last_unstale":"2024-04-24T23:11:49.339617+0000","last_undegraded":"2024-04-24T23:11:49.339617+0000","last_fullsized":"2024-04-24T23:11:49.339617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:21.817319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331454+0000","last_change":"2024-04-24T23:11:01.214041+0000","last_active":"2024-04-24T23:11:49.331454+0000","last_peered":"2024-04-24T23:11:49.331454+0000","last_clean":"2024-04-24T23:11:49.331454+0000","last_became_active":"2024-04-24T23:11:01.213639+0000","last_became_peered":"2024-04-24T23:11:01.213639+0000","last_unstale":"2024-04-24T23:11:49.331454+0000","last_undegraded":"2024-04-24T23:11:49.331454+0000","last_fullsized":"2024-04-24T23:11:49.331454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:08.819111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333807+0000","last_change":"2024-04-24T23:11:00.303790+0000","last_active":"2024-04-24T23:11:49.333807+0000","last_peered":"2024-04-24T23:11:49.333807+0000","last_clean":"2024-04-24T23:11:49.333807+0000","last_became_active":"2024-04-24T23:11:00.303381+0000","last_became_peered":"2024-04-24T23:11:00.303381+0000","last_unstale":"2024-04-24T23:11:49.333807+0000","last_undegraded":"2024-04-24T23:11:49.333807+0000","last_fullsized":"2024-04-24T23:11:49.333807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:18.725608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329573+0000","last_change":"2024-04-24T23:11:01.219575+0000","last_active":"2024-04-24T23:11:49.329573+0000","last_peered":"2024-04-24T23:11:49.329573+0000","last_clean":"2024-04-24T23:11:49.329573+0000","last_became_active":"2024-04-24T23:11:01.218750+0000","last_became_peered":"2024-04-24T23:11:01.218750+0000","last_unstale":"2024-04-24T23:11:49.329573+0000","last_undegraded":"2024-04-24T23:11:49.329573+0000","last_fullsized":"2024-04-24T23:11:49.329573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:00.945493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333162+0000","last_change":"2024-04-24T23:11:00.310468+0000","last_active":"2024-04-24T23:11:49.333162+0000","last_peered":"2024-04-24T23:11:49.333162+0000","last_clean":"2024-04-24T23:11:49.333162+0000","last_became_active":"2024-04-24T23:11:00.310176+0000","last_became_peered":"2024-04-24T23:11:00.310176+0000","last_unstale":"2024-04-24T23:11:49.333162+0000","last_undegraded":"2024-04-24T23:11:49.333162+0000","last_fullsized":"2024-04-24T23:11:49.333162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:25.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329819+0000","last_change":"2024-04-24T23:11:01.219593+0000","last_active":"2024-04-24T23:11:49.329819+0000","last_peered":"2024-04-24T23:11:49.329819+0000","last_clean":"2024-04-24T23:11:49.329819+0000","last_became_active":"2024-04-24T23:11:01.219247+0000","last_became_peered":"2024-04-24T23:11:01.219247+0000","last_unstale":"2024-04-24T23:11:49.329819+0000","last_undegraded":"2024-04-24T23:11:49.329819+0000","last_fullsized":"2024-04-24T23:11:49.329819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:12.605797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335957+0000","last_change":"2024-04-24T23:11:00.305124+0000","last_active":"2024-04-24T23:11:49.335957+0000","last_peered":"2024-04-24T23:11:49.335957+0000","last_clean":"2024-04-24T23:11:49.335957+0000","last_became_active":"2024-04-24T23:11:00.301468+0000","last_became_peered":"2024-04-24T23:11:00.301468+0000","last_unstale":"2024-04-24T23:11:49.335957+0000","last_undegraded":"2024-04-24T23:11:49.335957+0000","last_fullsized":"2024-04-24T23:11:49.335957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:58.451956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329501+0000","last_change":"2024-04-24T23:11:01.209967+0000","last_active":"2024-04-24T23:11:49.329501+0000","last_peered":"2024-04-24T23:11:49.329501+0000","last_clean":"2024-04-24T23:11:49.329501+0000","last_became_active":"2024-04-24T23:11:01.209607+0000","last_became_peered":"2024-04-24T23:11:01.209607+0000","last_unstale":"2024-04-24T23:11:49.329501+0000","last_undegraded":"2024-04-24T23:11:49.329501+0000","last_fullsized":"2024-04-24T23:11:49.329501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:02.840211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334745+0000","last_change":"2024-04-24T23:11:00.308200+0000","last_active":"2024-04-24T23:11:49.334745+0000","last_peered":"2024-04-24T23:11:49.334745+0000","last_clean":"2024-04-24T23:11:49.334745+0000","last_became_active":"2024-04-24T23:11:00.306369+0000","last_became_peered":"2024-04-24T23:11:00.306369+0000","last_unstale":"2024-04-24T23:11:49.334745+0000","last_undegraded":"2024-04-24T23:11:49.334745+0000","last_fullsized":"2024-04-24T23:11:49.334745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:24.462361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329883+0000","last_change":"2024-04-24T23:11:01.207706+0000","last_active":"2024-04-24T23:11:49.329883+0000","last_peered":"2024-04-24T23:11:49.329883+0000","last_clean":"2024-04-24T23:11:49.329883+0000","last_became_active":"2024-04-24T23:11:01.207357+0000","last_became_peered":"2024-04-24T23:11:01.207357+0000","last_unstale":"2024-04-24T23:11:49.329883+0000","last_undegraded":"2024-04-24T23:11:49.329883+0000","last_fullsized":"2024-04-24T23:11:49.329883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:46.785844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329606+0000","last_change":"2024-04-24T23:11:01.214886+0000","last_active":"2024-04-24T23:11:49.329606+0000","last_peered":"2024-04-24T23:11:49.329606+0000","last_clean":"2024-04-24T23:11:49.329606+0000","last_became_active":"2024-04-24T23:11:01.213671+0000","last_became_peered":"2024-04-24T23:11:01.213671+0000","last_unstale":"2024-04-24T23:11:49.329606+0000","last_undegraded":"2024-04-24T23:11:49.329606+0000","last_fullsized":"2024-04-24T23:11:49.329606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:44.777437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332752+0000","last_change":"2024-04-24T23:11:01.203578+0000","last_active":"2024-04-24T23:11:49.332752+0000","last_peered":"2024-04-24T23:11:49.332752+0000","last_clean":"2024-04-24T23:11:49.332752+0000","last_became_active":"2024-04-24T23:11:01.203250+0000","last_became_peered":"2024-04-24T23:11:01.203250+0000","last_unstale":"2024-04-24T23:11:49.332752+0000","last_undegraded":"2024-04-24T23:11:49.332752+0000","last_fullsized":"2024-04-24T23:11:49.332752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:39.050149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332243+0000","last_change":"2024-04-24T23:11:01.209641+0000","last_active":"2024-04-24T23:11:49.332243+0000","last_peered":"2024-04-24T23:11:49.332243+0000","last_clean":"2024-04-24T23:11:49.332243+0000","last_became_active":"2024-04-24T23:11:01.209110+0000","last_became_peered":"2024-04-24T23:11:01.209110+0000","last_unstale":"2024-04-24T23:11:49.332243+0000","last_undegraded":"2024-04-24T23:11:49.332243+0000","last_fullsized":"2024-04-24T23:11:49.332243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:14.709926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328727+0000","last_change":"2024-04-24T23:11:01.213894+0000","last_active":"2024-04-24T23:11:49.328727+0000","last_peered":"2024-04-24T23:11:49.328727+0000","last_clean":"2024-04-24T23:11:49.328727+0000","last_became_active":"2024-04-24T23:11:01.213461+0000","last_became_peered":"2024-04-24T23:11:01.213461+0000","last_unstale":"2024-04-24T23:11:49.328727+0000","last_undegraded":"2024-04-24T23:11:49.328727+0000","last_fullsized":"2024-04-24T23:11:49.328727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:00.786413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334742+0000","last_change":"2024-04-24T23:11:01.206556+0000","last_active":"2024-04-24T23:11:49.334742+0000","last_peered":"2024-04-24T23:11:49.334742+0000","last_clean":"2024-04-24T23:11:49.334742+0000","last_became_active":"2024-04-24T23:11:01.206178+0000","last_became_peered":"2024-04-24T23:11:01.206178+0000","last_unstale":"2024-04-24T23:11:49.334742+0000","last_undegraded":"2024-04-24T23:11:49.334742+0000","last_fullsized":"2024-04-24T23:11:49.334742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:56.765847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329846+0000","last_change":"2024-04-24T23:11:01.213174+0000","last_active":"2024-04-24T23:11:49.329846+0000","last_peered":"2024-04-24T23:11:49.329846+0000","last_clean":"2024-04-24T23:11:49.329846+0000","last_became_active":"2024-04-24T23:11:01.212815+0000","last_became_peered":"2024-04-24T23:11:01.212815+0000","last_unstale":"2024-04-24T23:11:49.329846+0000","last_undegraded":"2024-04-24T23:11:49.329846+0000","last_fullsized":"2024-04-24T23:11:49.329846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:09.281324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332745+0000","last_change":"2024-04-24T23:11:01.215030+0000","last_active":"2024-04-24T23:11:49.332745+0000","last_peered":"2024-04-24T23:11:49.332745+0000","last_clean":"2024-04-24T23:11:49.332745+0000","last_became_active":"2024-04-24T23:11:01.214386+0000","last_became_peered":"2024-04-24T23:11:01.214386+0000","last_unstale":"2024-04-24T23:11:49.332745+0000","last_undegraded":"2024-04-24T23:11:49.332745+0000","last_fullsized":"2024-04-24T23:11:49.332745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:10.633565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331839+0000","last_change":"2024-04-24T23:11:01.217748+0000","last_active":"2024-04-24T23:11:49.331839+0000","last_peered":"2024-04-24T23:11:49.331839+0000","last_clean":"2024-04-24T23:11:49.331839+0000","last_became_active":"2024-04-24T23:11:01.217403+0000","last_became_peered":"2024-04-24T23:11:01.217403+0000","last_unstale":"2024-04-24T23:11:49.331839+0000","last_undegraded":"2024-04-24T23:11:49.331839+0000","last_fullsized":"2024-04-24T23:11:49.331839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:29.569179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335433+0000","last_change":"2024-04-24T23:11:01.208894+0000","last_active":"2024-04-24T23:11:49.335433+0000","last_peered":"2024-04-24T23:11:49.335433+0000","last_clean":"2024-04-24T23:11:49.335433+0000","last_became_active":"2024-04-24T23:11:01.208576+0000","last_became_peered":"2024-04-24T23:11:01.208576+0000","last_unstale":"2024-04-24T23:11:49.335433+0000","last_undegraded":"2024-04-24T23:11:49.335433+0000","last_fullsized":"2024-04-24T23:11:49.335433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:13.967021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331335+0000","last_change":"2024-04-24T23:11:01.218561+0000","last_active":"2024-04-24T23:11:49.331335+0000","last_peered":"2024-04-24T23:11:49.331335+0000","last_clean":"2024-04-24T23:11:49.331335+0000","last_became_active":"2024-04-24T23:11:01.218261+0000","last_became_peered":"2024-04-24T23:11:01.218261+0000","last_unstale":"2024-04-24T23:11:49.331335+0000","last_undegraded":"2024-04-24T23:11:49.331335+0000","last_fullsized":"2024-04-24T23:11:49.331335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:49.677225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335228+0000","last_change":"2024-04-24T23:11:01.214868+0000","last_active":"2024-04-24T23:11:49.335228+0000","last_peered":"2024-04-24T23:11:49.335228+0000","last_clean":"2024-04-24T23:11:49.335228+0000","last_became_active":"2024-04-24T23:11:01.214582+0000","last_became_peered":"2024-04-24T23:11:01.214582+0000","last_unstale":"2024-04-24T23:11:49.335228+0000","last_undegraded":"2024-04-24T23:11:49.335228+0000","last_fullsized":"2024-04-24T23:11:49.335228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:48.963896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334444+0000","last_change":"2024-04-24T23:11:01.212348+0000","last_active":"2024-04-24T23:11:49.334444+0000","last_peered":"2024-04-24T23:11:49.334444+0000","last_clean":"2024-04-24T23:11:49.334444+0000","last_became_active":"2024-04-24T23:11:01.211802+0000","last_became_peered":"2024-04-24T23:11:01.211802+0000","last_unstale":"2024-04-24T23:11:49.334444+0000","last_undegraded":"2024-04-24T23:11:49.334444+0000","last_fullsized":"2024-04-24T23:11:49.334444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:49.484245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329694+0000","last_change":"2024-04-24T23:11:01.209834+0000","last_active":"2024-04-24T23:11:49.329694+0000","last_peered":"2024-04-24T23:11:49.329694+0000","last_clean":"2024-04-24T23:11:49.329694+0000","last_became_active":"2024-04-24T23:11:01.209476+0000","last_became_peered":"2024-04-24T23:11:01.209476+0000","last_unstale":"2024-04-24T23:11:49.329694+0000","last_undegraded":"2024-04-24T23:11:49.329694+0000","last_fullsized":"2024-04-24T23:11:49.329694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:20.036684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244416,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63304,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":440,"num_read_kb":289,"num_write":154,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":58912,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":189912,"data_compressed":13352,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":460802,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1111586,"data_compressed":43319,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":461038,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":460958,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1053279,"data_compressed":41436,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":460427,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":519132,"data_compressed":7700,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":461698,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":461961,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":460738,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":520167,"data_compressed":15403,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":1052572,"data_compressed":41436,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":58912,"data_compressed":1883,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":59947,"data_compressed":9586,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60881,"data_compressed":1883,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:12:17.959 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:12:18.202 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:18.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:18 smithi102 ceph-mon[29216]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:18.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:18 smithi161 ceph-mon[32709]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:18.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:18 smithi183 ceph-mon[33137]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:19 smithi102 ceph-mon[29216]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:19 smithi102 ceph-mon[29216]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-24T23:12:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:19 smithi161 ceph-mon[32709]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:19 smithi161 ceph-mon[32709]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-24T23:12:19.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:19 smithi183 ceph-mon[33137]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:19.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:19 smithi183 ceph-mon[33137]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-24T23:12:19.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.885+0000 7fb80c86b700 1 -- 172.21.15.102:0/1919102442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 msgr2=0x7fb80810f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.885+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1919102442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 0x7fb80810f110 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb80000b0d0 tx=0x7fb80000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.885+0000 7fb80c86b700 1 -- 172.21.15.102:0/1919102442 shutdown_connections 2024-04-24T23:12:19.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.885+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1919102442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 0x7fb80810f110 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.885+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1919102442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb808102550 0x7fb808102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.885+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1919102442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb808102f00 0x7fb808106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.885+0000 7fb80c86b700 1 -- 172.21.15.102:0/1919102442 >> 172.21.15.102:0/1919102442 conn(0x7fb8080793f0 msgr2=0x7fb808079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:19.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.885+0000 7fb80c86b700 1 -- 172.21.15.102:0/1919102442 shutdown_connections 2024-04-24T23:12:19.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.885+0000 7fb80c86b700 1 -- 172.21.15.102:0/1919102442 wait complete. 2024-04-24T23:12:19.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.886+0000 7fb80c86b700 1 Processor -- start 2024-04-24T23:12:19.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.886+0000 7fb80c86b700 1 -- start start 2024-04-24T23:12:19.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.886+0000 7fb80c86b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb808102550 0x7fb80819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:19.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.886+0000 7fb80c86b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb808102f00 0x7fb80819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:19.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.886+0000 7fb80c86b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 0x7fb808198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:19.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.886+0000 7fb80c86b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb808112450 con 0x7fb808107540 2024-04-24T23:12:19.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.886+0000 7fb80c86b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081122d0 con 0x7fb808102f00 2024-04-24T23:12:19.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.886+0000 7fb80c86b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081125d0 con 0x7fb808102550 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb806d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 0x7fb808198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb806d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 0x7fb808198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60742/0 (socket says 172.21.15.102:60742) 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb806d9d700 1 -- 172.21.15.102:0/1882369040 learned_addr learned my addr 172.21.15.102:0/1882369040 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb80659c700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb808102550 0x7fb80819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb805d9b700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb808102f00 0x7fb80819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb806d9d700 1 -- 172.21.15.102:0/1882369040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb808102550 msgr2=0x7fb80819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb806d9d700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb808102550 0x7fb80819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb806d9d700 1 -- 172.21.15.102:0/1882369040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb808102f00 msgr2=0x7fb80819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb806d9d700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb808102f00 0x7fb80819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb806d9d700 1 -- 172.21.15.102:0/1882369040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb808199050 con 0x7fb808107540 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb806d9d700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 0x7fb808198770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb800009fd0 tx=0x7fb800007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb7f77fe700 1 -- 172.21.15.102:0/1882369040 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80001f070 con 0x7fb808107540 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8081992e0 con 0x7fb808107540 2024-04-24T23:12:19.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb7f77fe700 1 -- 172.21.15.102:0/1882369040 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb800003c00 con 0x7fb808107540 2024-04-24T23:12:19.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb7f77fe700 1 -- 172.21.15.102:0/1882369040 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80001b820 con 0x7fb808107540 2024-04-24T23:12:19.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.887+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8081a5400 con 0x7fb808107540 2024-04-24T23:12:19.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.888+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb808103940 con 0x7fb808107540 2024-04-24T23:12:19.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.890+0000 7fb7f77fe700 1 -- 172.21.15.102:0/1882369040 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb800022070 con 0x7fb808107540 2024-04-24T23:12:19.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.891+0000 7fb7f77fe700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb7f00a2c20 0x7fb7f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:19.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.891+0000 7fb80659c700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb7f00a2c20 0x7fb7f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:19.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.891+0000 7fb7f77fe700 1 -- 172.21.15.102:0/1882369040 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb80000f020 con 0x7fb808107540 2024-04-24T23:12:19.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.891+0000 7fb80659c700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb7f00a2c20 0x7fb7f00a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc005fd0 tx=0x7fb7fc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:19.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:19.894+0000 7fb7f77fe700 1 -- 172.21.15.102:0/1882369040 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb800029020 con 0x7fb808107540 2024-04-24T23:12:20.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.088+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb808062670 con 0x7fb7f00a2c20 2024-04-24T23:12:20.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.095+0000 7fb7f77fe700 1 -- 172.21.15.102:0/1882369040 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258991 (secure 0 0 0) 0x7fb808062670 con 0x7fb7f00a2c20 2024-04-24T23:12:20.100 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:20.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb7f00a2c20 msgr2=0x7fb7f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:20.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb7f00a2c20 0x7fb7f00a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc005fd0 tx=0x7fb7fc00a560 comp rx=0 tx=0).stop 2024-04-24T23:12:20.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 msgr2=0x7fb808198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:20.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 0x7fb808198770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb800009fd0 tx=0x7fb800007ad0 comp rx=0 tx=0).stop 2024-04-24T23:12:20.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 shutdown_connections 2024-04-24T23:12:20.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb7f00a2c20 0x7fb7f00a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:20.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb808107540 0x7fb808198770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:20.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb808102550 0x7fb80819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:20.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 --2- 172.21.15.102:0/1882369040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb808102f00 0x7fb80819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:20.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 >> 172.21.15.102:0/1882369040 conn(0x7fb8080793f0 msgr2=0x7fb808100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:20.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 shutdown_connections 2024-04-24T23:12:20.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:20.097+0000 7fb80c86b700 1 -- 172.21.15.102:0/1882369040 wait complete. 2024-04-24T23:12:20.103 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-04-24T23:12:20.773 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-04-24T23:12:18.236210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653906,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":591,"num_read_kb":413,"num_write":337,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1239648,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356720,"kb_used_data":11660,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585104,"statfs":{"total":1151940427776,"available":1151575146496,"internally_reserved":0,"allocated":11939840,"data_stored":7484358,"data_compressed":190013,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003451"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.330059+0000","last_change":"2024-04-24T23:11:01.217761+0000","last_active":"2024-04-24T23:11:49.330059+0000","last_peered":"2024-04-24T23:11:49.330059+0000","last_clean":"2024-04-24T23:11:49.330059+0000","last_became_active":"2024-04-24T23:11:01.217330+0000","last_became_peered":"2024-04-24T23:11:01.217330+0000","last_unstale":"2024-04-24T23:11:49.330059+0000","last_undegraded":"2024-04-24T23:11:49.330059+0000","last_fullsized":"2024-04-24T23:11:49.330059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:48:10.690940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328775+0000","last_change":"2024-04-24T23:11:01.206722+0000","last_active":"2024-04-24T23:11:49.328775+0000","last_peered":"2024-04-24T23:11:49.328775+0000","last_clean":"2024-04-24T23:11:49.328775+0000","last_became_active":"2024-04-24T23:11:01.205573+0000","last_became_peered":"2024-04-24T23:11:01.205573+0000","last_unstale":"2024-04-24T23:11:49.328775+0000","last_undegraded":"2024-04-24T23:11:49.328775+0000","last_fullsized":"2024-04-24T23:11:49.328775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:01.757792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328830+0000","last_change":"2024-04-24T23:11:01.213159+0000","last_active":"2024-04-24T23:11:49.328830+0000","last_peered":"2024-04-24T23:11:49.328830+0000","last_clean":"2024-04-24T23:11:49.328830+0000","last_became_active":"2024-04-24T23:11:01.212811+0000","last_became_peered":"2024-04-24T23:11:01.212811+0000","last_unstale":"2024-04-24T23:11:49.328830+0000","last_undegraded":"2024-04-24T23:11:49.328830+0000","last_fullsized":"2024-04-24T23:11:49.328830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:02.053206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.711745+0000","last_change":"2024-04-24T23:11:01.213023+0000","last_active":"2024-04-24T23:11:49.711745+0000","last_peered":"2024-04-24T23:11:49.711745+0000","last_clean":"2024-04-24T23:11:49.711745+0000","last_became_active":"2024-04-24T23:11:01.212689+0000","last_became_peered":"2024-04-24T23:11:01.212689+0000","last_unstale":"2024-04-24T23:11:49.711745+0000","last_undegraded":"2024-04-24T23:11:49.711745+0000","last_fullsized":"2024-04-24T23:11:49.711745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:41.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333026+0000","last_change":"2024-04-24T23:11:01.210845+0000","last_active":"2024-04-24T23:11:49.333026+0000","last_peered":"2024-04-24T23:11:49.333026+0000","last_clean":"2024-04-24T23:11:49.333026+0000","last_became_active":"2024-04-24T23:11:01.210389+0000","last_became_peered":"2024-04-24T23:11:01.210389+0000","last_unstale":"2024-04-24T23:11:49.333026+0000","last_undegraded":"2024-04-24T23:11:49.333026+0000","last_fullsized":"2024-04-24T23:11:49.333026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:03.946294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332353+0000","last_change":"2024-04-24T23:11:01.203082+0000","last_active":"2024-04-24T23:11:49.332353+0000","last_peered":"2024-04-24T23:11:49.332353+0000","last_clean":"2024-04-24T23:11:49.332353+0000","last_became_active":"2024-04-24T23:11:01.202605+0000","last_became_peered":"2024-04-24T23:11:01.202605+0000","last_unstale":"2024-04-24T23:11:49.332353+0000","last_undegraded":"2024-04-24T23:11:49.332353+0000","last_fullsized":"2024-04-24T23:11:49.332353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:29.248361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329375+0000","last_change":"2024-04-24T23:11:01.213002+0000","last_active":"2024-04-24T23:11:49.329375+0000","last_peered":"2024-04-24T23:11:49.329375+0000","last_clean":"2024-04-24T23:11:49.329375+0000","last_became_active":"2024-04-24T23:11:01.212690+0000","last_became_peered":"2024-04-24T23:11:01.212690+0000","last_unstale":"2024-04-24T23:11:49.329375+0000","last_undegraded":"2024-04-24T23:11:49.329375+0000","last_fullsized":"2024-04-24T23:11:49.329375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:06.658975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.336156+0000","last_change":"2024-04-24T23:11:49.335234+0000","last_active":"2024-04-24T23:11:49.336156+0000","last_peered":"2024-04-24T23:11:49.336156+0000","last_clean":"2024-04-24T23:11:49.336156+0000","last_became_active":"2024-04-24T23:11:49.334777+0000","last_became_peered":"2024-04-24T23:11:49.334777+0000","last_unstale":"2024-04-24T23:11:49.336156+0000","last_undegraded":"2024-04-24T23:11:49.336156+0000","last_fullsized":"2024-04-24T23:11:49.336156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:41.254260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335531+0000","last_change":"2024-04-24T23:11:01.217638+0000","last_active":"2024-04-24T23:11:49.335531+0000","last_peered":"2024-04-24T23:11:49.335531+0000","last_clean":"2024-04-24T23:11:49.335531+0000","last_became_active":"2024-04-24T23:11:01.217288+0000","last_became_peered":"2024-04-24T23:11:01.217288+0000","last_unstale":"2024-04-24T23:11:49.335531+0000","last_undegraded":"2024-04-24T23:11:49.335531+0000","last_fullsized":"2024-04-24T23:11:49.335531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:24.128868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333154+0000","last_change":"2024-04-24T23:11:01.203188+0000","last_active":"2024-04-24T23:11:49.333154+0000","last_peered":"2024-04-24T23:11:49.333154+0000","last_clean":"2024-04-24T23:11:49.333154+0000","last_became_active":"2024-04-24T23:11:01.202763+0000","last_became_peered":"2024-04-24T23:11:01.202763+0000","last_unstale":"2024-04-24T23:11:49.333154+0000","last_undegraded":"2024-04-24T23:11:49.333154+0000","last_fullsized":"2024-04-24T23:11:49.333154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:04.414650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329773+0000","last_change":"2024-04-24T23:11:01.220641+0000","last_active":"2024-04-24T23:11:49.329773+0000","last_peered":"2024-04-24T23:11:49.329773+0000","last_clean":"2024-04-24T23:11:49.329773+0000","last_became_active":"2024-04-24T23:11:01.220344+0000","last_became_peered":"2024-04-24T23:11:01.220344+0000","last_unstale":"2024-04-24T23:11:49.329773+0000","last_undegraded":"2024-04-24T23:11:49.329773+0000","last_fullsized":"2024-04-24T23:11:49.329773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:27:44.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335881+0000","last_change":"2024-04-24T23:11:49.334950+0000","last_active":"2024-04-24T23:11:49.335881+0000","last_peered":"2024-04-24T23:11:49.335881+0000","last_clean":"2024-04-24T23:11:49.335881+0000","last_became_active":"2024-04-24T23:11:49.334585+0000","last_became_peered":"2024-04-24T23:11:49.334585+0000","last_unstale":"2024-04-24T23:11:49.335881+0000","last_undegraded":"2024-04-24T23:11:49.335881+0000","last_fullsized":"2024-04-24T23:11:49.335881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:05.223004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334617+0000","last_change":"2024-04-24T23:11:01.205763+0000","last_active":"2024-04-24T23:11:49.334617+0000","last_peered":"2024-04-24T23:11:49.334617+0000","last_clean":"2024-04-24T23:11:49.334617+0000","last_became_active":"2024-04-24T23:11:01.205402+0000","last_became_peered":"2024-04-24T23:11:01.205402+0000","last_unstale":"2024-04-24T23:11:49.334617+0000","last_undegraded":"2024-04-24T23:11:49.334617+0000","last_fullsized":"2024-04-24T23:11:49.334617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:40.976719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328699+0000","last_change":"2024-04-24T23:11:01.209466+0000","last_active":"2024-04-24T23:11:49.328699+0000","last_peered":"2024-04-24T23:11:49.328699+0000","last_clean":"2024-04-24T23:11:49.328699+0000","last_became_active":"2024-04-24T23:11:01.208946+0000","last_became_peered":"2024-04-24T23:11:01.208946+0000","last_unstale":"2024-04-24T23:11:49.328699+0000","last_undegraded":"2024-04-24T23:11:49.328699+0000","last_fullsized":"2024-04-24T23:11:49.328699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:07.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328609+0000","last_change":"2024-04-24T23:11:01.215170+0000","last_active":"2024-04-24T23:11:49.328609+0000","last_peered":"2024-04-24T23:11:49.328609+0000","last_clean":"2024-04-24T23:11:49.328609+0000","last_became_active":"2024-04-24T23:11:01.214867+0000","last_became_peered":"2024-04-24T23:11:01.214867+0000","last_unstale":"2024-04-24T23:11:49.328609+0000","last_undegraded":"2024-04-24T23:11:49.328609+0000","last_fullsized":"2024-04-24T23:11:49.328609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.830234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335026+0000","last_change":"2024-04-24T23:11:01.216691+0000","last_active":"2024-04-24T23:11:49.335026+0000","last_peered":"2024-04-24T23:11:49.335026+0000","last_clean":"2024-04-24T23:11:49.335026+0000","last_became_active":"2024-04-24T23:11:01.216407+0000","last_became_peered":"2024-04-24T23:11:01.216407+0000","last_unstale":"2024-04-24T23:11:49.335026+0000","last_undegraded":"2024-04-24T23:11:49.335026+0000","last_fullsized":"2024-04-24T23:11:49.335026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:03.284374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.330072+0000","last_change":"2024-04-24T23:11:01.217764+0000","last_active":"2024-04-24T23:11:49.330072+0000","last_peered":"2024-04-24T23:11:49.330072+0000","last_clean":"2024-04-24T23:11:49.330072+0000","last_became_active":"2024-04-24T23:11:01.217424+0000","last_became_peered":"2024-04-24T23:11:01.217424+0000","last_unstale":"2024-04-24T23:11:49.330072+0000","last_undegraded":"2024-04-24T23:11:49.330072+0000","last_fullsized":"2024-04-24T23:11:49.330072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:11.517317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326384+0000","last_change":"2024-04-24T23:11:01.214351+0000","last_active":"2024-04-24T23:11:49.326384+0000","last_peered":"2024-04-24T23:11:49.326384+0000","last_clean":"2024-04-24T23:11:49.326384+0000","last_became_active":"2024-04-24T23:11:01.214034+0000","last_became_peered":"2024-04-24T23:11:01.214034+0000","last_unstale":"2024-04-24T23:11:49.326384+0000","last_undegraded":"2024-04-24T23:11:49.326384+0000","last_fullsized":"2024-04-24T23:11:49.326384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:26.105888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332418+0000","last_change":"2024-04-24T23:11:01.212836+0000","last_active":"2024-04-24T23:11:49.332418+0000","last_peered":"2024-04-24T23:11:49.332418+0000","last_clean":"2024-04-24T23:11:49.332418+0000","last_became_active":"2024-04-24T23:11:01.209223+0000","last_became_peered":"2024-04-24T23:11:01.209223+0000","last_unstale":"2024-04-24T23:11:49.332418+0000","last_undegraded":"2024-04-24T23:11:49.332418+0000","last_fullsized":"2024-04-24T23:11:49.332418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:40.100399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326207+0000","last_change":"2024-04-24T23:11:00.302055+0000","last_active":"2024-04-24T23:11:49.326207+0000","last_peered":"2024-04-24T23:11:49.326207+0000","last_clean":"2024-04-24T23:11:49.326207+0000","last_became_active":"2024-04-24T23:11:00.301726+0000","last_became_peered":"2024-04-24T23:11:00.301726+0000","last_unstale":"2024-04-24T23:11:49.326207+0000","last_undegraded":"2024-04-24T23:11:49.326207+0000","last_fullsized":"2024-04-24T23:11:49.326207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:54.039099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331797+0000","last_change":"2024-04-24T23:11:01.219527+0000","last_active":"2024-04-24T23:11:49.331797+0000","last_peered":"2024-04-24T23:11:49.331797+0000","last_clean":"2024-04-24T23:11:49.331797+0000","last_became_active":"2024-04-24T23:11:01.219196+0000","last_became_peered":"2024-04-24T23:11:01.219196+0000","last_unstale":"2024-04-24T23:11:49.331797+0000","last_undegraded":"2024-04-24T23:11:49.331797+0000","last_fullsized":"2024-04-24T23:11:49.331797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:58.584402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331811+0000","last_change":"2024-04-24T23:11:00.309260+0000","last_active":"2024-04-24T23:11:49.331811+0000","last_peered":"2024-04-24T23:11:49.331811+0000","last_clean":"2024-04-24T23:11:49.331811+0000","last_became_active":"2024-04-24T23:11:00.308969+0000","last_became_peered":"2024-04-24T23:11:00.308969+0000","last_unstale":"2024-04-24T23:11:49.331811+0000","last_undegraded":"2024-04-24T23:11:49.331811+0000","last_fullsized":"2024-04-24T23:11:49.331811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:23.137469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328936+0000","last_change":"2024-04-24T23:11:01.216515+0000","last_active":"2024-04-24T23:11:49.328936+0000","last_peered":"2024-04-24T23:11:49.328936+0000","last_clean":"2024-04-24T23:11:49.328936+0000","last_became_active":"2024-04-24T23:11:01.216168+0000","last_became_peered":"2024-04-24T23:11:01.216168+0000","last_unstale":"2024-04-24T23:11:49.328936+0000","last_undegraded":"2024-04-24T23:11:49.328936+0000","last_fullsized":"2024-04-24T23:11:49.328936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:05.229365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333570+0000","last_change":"2024-04-24T23:11:00.302558+0000","last_active":"2024-04-24T23:11:49.333570+0000","last_peered":"2024-04-24T23:11:49.333570+0000","last_clean":"2024-04-24T23:11:49.333570+0000","last_became_active":"2024-04-24T23:11:00.302162+0000","last_became_peered":"2024-04-24T23:11:00.302162+0000","last_unstale":"2024-04-24T23:11:49.333570+0000","last_undegraded":"2024-04-24T23:11:49.333570+0000","last_fullsized":"2024-04-24T23:11:49.333570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:40:52.623679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335407+0000","last_change":"2024-04-24T23:11:01.222129+0000","last_active":"2024-04-24T23:11:49.335407+0000","last_peered":"2024-04-24T23:11:49.335407+0000","last_clean":"2024-04-24T23:11:49.335407+0000","last_became_active":"2024-04-24T23:11:01.221795+0000","last_became_peered":"2024-04-24T23:11:01.221795+0000","last_unstale":"2024-04-24T23:11:49.335407+0000","last_undegraded":"2024-04-24T23:11:49.335407+0000","last_fullsized":"2024-04-24T23:11:49.335407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:21.417562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334138+0000","last_change":"2024-04-24T23:11:00.308586+0000","last_active":"2024-04-24T23:11:49.334138+0000","last_peered":"2024-04-24T23:11:49.334138+0000","last_clean":"2024-04-24T23:11:49.334138+0000","last_became_active":"2024-04-24T23:11:00.305054+0000","last_became_peered":"2024-04-24T23:11:00.305054+0000","last_unstale":"2024-04-24T23:11:49.334138+0000","last_undegraded":"2024-04-24T23:11:49.334138+0000","last_fullsized":"2024-04-24T23:11:49.334138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:11.371829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.338667+0000","last_change":"2024-04-24T23:11:49.338667+0000","last_active":"2024-04-24T23:11:49.338667+0000","last_peered":"2024-04-24T23:11:49.338667+0000","last_clean":"2024-04-24T23:11:49.338667+0000","last_became_active":"2024-04-24T23:11:49.338218+0000","last_became_peered":"2024-04-24T23:11:49.338218+0000","last_unstale":"2024-04-24T23:11:49.338667+0000","last_undegraded":"2024-04-24T23:11:49.338667+0000","last_fullsized":"2024-04-24T23:11:49.338667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:42.061058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:14.974568+0000","last_change":"2024-04-24T23:11:00.302389+0000","last_active":"2024-04-24T23:12:14.974568+0000","last_peered":"2024-04-24T23:12:14.974568+0000","last_clean":"2024-04-24T23:12:14.974568+0000","last_became_active":"2024-04-24T23:11:00.302061+0000","last_became_peered":"2024-04-24T23:11:00.302061+0000","last_unstale":"2024-04-24T23:12:14.974568+0000","last_undegraded":"2024-04-24T23:12:14.974568+0000","last_fullsized":"2024-04-24T23:12:14.974568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:28.437391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333185+0000","last_change":"2024-04-24T23:11:01.202389+0000","last_active":"2024-04-24T23:11:49.333185+0000","last_peered":"2024-04-24T23:11:49.333185+0000","last_clean":"2024-04-24T23:11:49.333185+0000","last_became_active":"2024-04-24T23:11:01.201986+0000","last_became_peered":"2024-04-24T23:11:01.201986+0000","last_unstale":"2024-04-24T23:11:49.333185+0000","last_undegraded":"2024-04-24T23:11:49.333185+0000","last_fullsized":"2024-04-24T23:11:49.333185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:25.743284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.715664+0000","last_change":"2024-04-24T23:11:49.715664+0000","last_active":"2024-04-24T23:11:49.715664+0000","last_peered":"2024-04-24T23:11:49.715664+0000","last_clean":"2024-04-24T23:11:49.715664+0000","last_became_active":"2024-04-24T23:11:49.712054+0000","last_became_peered":"2024-04-24T23:11:49.712054+0000","last_unstale":"2024-04-24T23:11:49.715664+0000","last_undegraded":"2024-04-24T23:11:49.715664+0000","last_fullsized":"2024-04-24T23:11:49.715664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:51.167456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.336088+0000","last_change":"2024-04-24T23:11:01.214868+0000","last_active":"2024-04-24T23:11:49.336088+0000","last_peered":"2024-04-24T23:11:49.336088+0000","last_clean":"2024-04-24T23:11:49.336088+0000","last_became_active":"2024-04-24T23:11:01.211366+0000","last_became_peered":"2024-04-24T23:11:01.211366+0000","last_unstale":"2024-04-24T23:11:49.336088+0000","last_undegraded":"2024-04-24T23:11:49.336088+0000","last_fullsized":"2024-04-24T23:11:49.336088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:21.595136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334151+0000","last_change":"2024-04-24T23:11:00.302050+0000","last_active":"2024-04-24T23:11:49.334151+0000","last_peered":"2024-04-24T23:11:49.334151+0000","last_clean":"2024-04-24T23:11:49.334151+0000","last_became_active":"2024-04-24T23:11:00.301740+0000","last_became_peered":"2024-04-24T23:11:00.301740+0000","last_unstale":"2024-04-24T23:11:49.334151+0000","last_undegraded":"2024-04-24T23:11:49.334151+0000","last_fullsized":"2024-04-24T23:11:49.334151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:34.994687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331323+0000","last_change":"2024-04-24T23:11:01.211470+0000","last_active":"2024-04-24T23:11:49.331323+0000","last_peered":"2024-04-24T23:11:49.331323+0000","last_clean":"2024-04-24T23:11:49.331323+0000","last_became_active":"2024-04-24T23:11:01.209482+0000","last_became_peered":"2024-04-24T23:11:01.209482+0000","last_unstale":"2024-04-24T23:11:49.331323+0000","last_undegraded":"2024-04-24T23:11:49.331323+0000","last_fullsized":"2024-04-24T23:11:49.331323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:01.407263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335344+0000","last_change":"2024-04-24T23:11:00.305671+0000","last_active":"2024-04-24T23:11:49.335344+0000","last_peered":"2024-04-24T23:11:49.335344+0000","last_clean":"2024-04-24T23:11:49.335344+0000","last_became_active":"2024-04-24T23:11:00.303016+0000","last_became_peered":"2024-04-24T23:11:00.303016+0000","last_unstale":"2024-04-24T23:11:49.335344+0000","last_undegraded":"2024-04-24T23:11:49.335344+0000","last_fullsized":"2024-04-24T23:11:49.335344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:33.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326156+0000","last_change":"2024-04-24T23:11:01.206269+0000","last_active":"2024-04-24T23:11:49.326156+0000","last_peered":"2024-04-24T23:11:49.326156+0000","last_clean":"2024-04-24T23:11:49.326156+0000","last_became_active":"2024-04-24T23:11:01.205941+0000","last_became_peered":"2024-04-24T23:11:01.205941+0000","last_unstale":"2024-04-24T23:11:49.326156+0000","last_undegraded":"2024-04-24T23:11:49.326156+0000","last_fullsized":"2024-04-24T23:11:49.326156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:28.955247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:13.972352+0000","last_change":"2024-04-24T23:11:00.306071+0000","last_active":"2024-04-24T23:12:13.972352+0000","last_peered":"2024-04-24T23:12:13.972352+0000","last_clean":"2024-04-24T23:12:13.972352+0000","last_became_active":"2024-04-24T23:11:00.305751+0000","last_became_peered":"2024-04-24T23:11:00.305751+0000","last_unstale":"2024-04-24T23:12:13.972352+0000","last_undegraded":"2024-04-24T23:12:13.972352+0000","last_fullsized":"2024-04-24T23:12:13.972352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:33:30.675323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335735+0000","last_change":"2024-04-24T23:11:01.222118+0000","last_active":"2024-04-24T23:11:49.335735+0000","last_peered":"2024-04-24T23:11:49.335735+0000","last_clean":"2024-04-24T23:11:49.335735+0000","last_became_active":"2024-04-24T23:11:01.221830+0000","last_became_peered":"2024-04-24T23:11:01.221830+0000","last_unstale":"2024-04-24T23:11:49.335735+0000","last_undegraded":"2024-04-24T23:11:49.335735+0000","last_fullsized":"2024-04-24T23:11:49.335735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:18.937230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329800+0000","last_change":"2024-04-24T23:11:00.308662+0000","last_active":"2024-04-24T23:11:49.329800+0000","last_peered":"2024-04-24T23:11:49.329800+0000","last_clean":"2024-04-24T23:11:49.329800+0000","last_became_active":"2024-04-24T23:11:00.308355+0000","last_became_peered":"2024-04-24T23:11:00.308355+0000","last_unstale":"2024-04-24T23:11:49.329800+0000","last_undegraded":"2024-04-24T23:11:49.329800+0000","last_fullsized":"2024-04-24T23:11:49.329800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:15.228277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.338279+0000","last_change":"2024-04-24T23:11:49.338279+0000","last_active":"2024-04-24T23:11:49.338279+0000","last_peered":"2024-04-24T23:11:49.338279+0000","last_clean":"2024-04-24T23:11:49.338279+0000","last_became_active":"2024-04-24T23:11:49.338030+0000","last_became_peered":"2024-04-24T23:11:49.338030+0000","last_unstale":"2024-04-24T23:11:49.338279+0000","last_undegraded":"2024-04-24T23:11:49.338279+0000","last_fullsized":"2024-04-24T23:11:49.338279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:30.867560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.336108+0000","last_change":"2024-04-24T23:11:00.308114+0000","last_active":"2024-04-24T23:11:49.336108+0000","last_peered":"2024-04-24T23:11:49.336108+0000","last_clean":"2024-04-24T23:11:49.336108+0000","last_became_active":"2024-04-24T23:11:00.306632+0000","last_became_peered":"2024-04-24T23:11:00.306632+0000","last_unstale":"2024-04-24T23:11:49.336108+0000","last_undegraded":"2024-04-24T23:11:49.336108+0000","last_fullsized":"2024-04-24T23:11:49.336108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:19.298736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335554+0000","last_change":"2024-04-24T23:11:01.219789+0000","last_active":"2024-04-24T23:11:49.335554+0000","last_peered":"2024-04-24T23:11:49.335554+0000","last_clean":"2024-04-24T23:11:49.335554+0000","last_became_active":"2024-04-24T23:11:01.219498+0000","last_became_peered":"2024-04-24T23:11:01.219498+0000","last_unstale":"2024-04-24T23:11:49.335554+0000","last_undegraded":"2024-04-24T23:11:49.335554+0000","last_fullsized":"2024-04-24T23:11:49.335554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:26.076594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:14.755395+0000","last_change":"2024-04-24T23:11:49.721720+0000","last_active":"2024-04-24T23:12:14.755395+0000","last_peered":"2024-04-24T23:12:14.755395+0000","last_clean":"2024-04-24T23:12:14.755395+0000","last_became_active":"2024-04-24T23:11:49.712796+0000","last_became_peered":"2024-04-24T23:11:49.712796+0000","last_unstale":"2024-04-24T23:12:14.755395+0000","last_undegraded":"2024-04-24T23:12:14.755395+0000","last_fullsized":"2024-04-24T23:12:14.755395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:36.431137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":147,"num_read_kb":130,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":58912,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333824+0000","last_change":"2024-04-24T23:11:01.204997+0000","last_active":"2024-04-24T23:11:49.333824+0000","last_peered":"2024-04-24T23:11:49.333824+0000","last_clean":"2024-04-24T23:11:49.333824+0000","last_became_active":"2024-04-24T23:11:01.204645+0000","last_became_peered":"2024-04-24T23:11:01.204645+0000","last_unstale":"2024-04-24T23:11:49.333824+0000","last_undegraded":"2024-04-24T23:11:49.333824+0000","last_fullsized":"2024-04-24T23:11:49.333824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:07.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332025+0000","last_change":"2024-04-24T23:11:00.302465+0000","last_active":"2024-04-24T23:11:49.332025+0000","last_peered":"2024-04-24T23:11:49.332025+0000","last_clean":"2024-04-24T23:11:49.332025+0000","last_became_active":"2024-04-24T23:11:00.302162+0000","last_became_peered":"2024-04-24T23:11:00.302162+0000","last_unstale":"2024-04-24T23:11:49.332025+0000","last_undegraded":"2024-04-24T23:11:49.332025+0000","last_fullsized":"2024-04-24T23:11:49.332025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:51.352872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334477+0000","last_change":"2024-04-24T23:11:01.206588+0000","last_active":"2024-04-24T23:11:49.334477+0000","last_peered":"2024-04-24T23:11:49.334477+0000","last_clean":"2024-04-24T23:11:49.334477+0000","last_became_active":"2024-04-24T23:11:01.206258+0000","last_became_peered":"2024-04-24T23:11:01.206258+0000","last_unstale":"2024-04-24T23:11:49.334477+0000","last_undegraded":"2024-04-24T23:11:49.334477+0000","last_fullsized":"2024-04-24T23:11:49.334477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:26.857203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335184+0000","last_change":"2024-04-24T23:11:00.305179+0000","last_active":"2024-04-24T23:11:49.335184+0000","last_peered":"2024-04-24T23:11:49.335184+0000","last_clean":"2024-04-24T23:11:49.335184+0000","last_became_active":"2024-04-24T23:11:00.302951+0000","last_became_peered":"2024-04-24T23:11:00.302951+0000","last_unstale":"2024-04-24T23:11:49.335184+0000","last_undegraded":"2024-04-24T23:11:49.335184+0000","last_fullsized":"2024-04-24T23:11:49.335184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:33.278498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329628+0000","last_change":"2024-04-24T23:11:01.211062+0000","last_active":"2024-04-24T23:11:49.329628+0000","last_peered":"2024-04-24T23:11:49.329628+0000","last_clean":"2024-04-24T23:11:49.329628+0000","last_became_active":"2024-04-24T23:11:01.210752+0000","last_became_peered":"2024-04-24T23:11:01.210752+0000","last_unstale":"2024-04-24T23:11:49.329628+0000","last_undegraded":"2024-04-24T23:11:49.329628+0000","last_fullsized":"2024-04-24T23:11:49.329628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:54.790212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:14.987157+0000","last_change":"2024-04-24T23:11:00.305690+0000","last_active":"2024-04-24T23:12:14.987157+0000","last_peered":"2024-04-24T23:12:14.987157+0000","last_clean":"2024-04-24T23:12:14.987157+0000","last_became_active":"2024-04-24T23:11:00.305386+0000","last_became_peered":"2024-04-24T23:11:00.305386+0000","last_unstale":"2024-04-24T23:12:14.987157+0000","last_undegraded":"2024-04-24T23:12:14.987157+0000","last_fullsized":"2024-04-24T23:12:14.987157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:39.898085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329244+0000","last_change":"2024-04-24T23:11:01.208916+0000","last_active":"2024-04-24T23:11:49.329244+0000","last_peered":"2024-04-24T23:11:49.329244+0000","last_clean":"2024-04-24T23:11:49.329244+0000","last_became_active":"2024-04-24T23:11:01.208593+0000","last_became_peered":"2024-04-24T23:11:01.208593+0000","last_unstale":"2024-04-24T23:11:49.329244+0000","last_undegraded":"2024-04-24T23:11:49.329244+0000","last_fullsized":"2024-04-24T23:11:49.329244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:58.442365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332974+0000","last_change":"2024-04-24T23:11:00.309608+0000","last_active":"2024-04-24T23:11:49.332974+0000","last_peered":"2024-04-24T23:11:49.332974+0000","last_clean":"2024-04-24T23:11:49.332974+0000","last_became_active":"2024-04-24T23:11:00.309310+0000","last_became_peered":"2024-04-24T23:11:00.309310+0000","last_unstale":"2024-04-24T23:11:49.332974+0000","last_undegraded":"2024-04-24T23:11:49.332974+0000","last_fullsized":"2024-04-24T23:11:49.332974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:23.330558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335086+0000","last_change":"2024-04-24T23:11:01.212376+0000","last_active":"2024-04-24T23:11:49.335086+0000","last_peered":"2024-04-24T23:11:49.335086+0000","last_clean":"2024-04-24T23:11:49.335086+0000","last_became_active":"2024-04-24T23:11:01.212043+0000","last_became_peered":"2024-04-24T23:11:01.212043+0000","last_unstale":"2024-04-24T23:11:49.335086+0000","last_undegraded":"2024-04-24T23:11:49.335086+0000","last_fullsized":"2024-04-24T23:11:49.335086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:54.025829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332970+0000","last_change":"2024-04-24T23:11:00.305483+0000","last_active":"2024-04-24T23:11:49.332970+0000","last_peered":"2024-04-24T23:11:49.332970+0000","last_clean":"2024-04-24T23:11:49.332970+0000","last_became_active":"2024-04-24T23:11:00.305186+0000","last_became_peered":"2024-04-24T23:11:00.305186+0000","last_unstale":"2024-04-24T23:11:49.332970+0000","last_undegraded":"2024-04-24T23:11:49.332970+0000","last_fullsized":"2024-04-24T23:11:49.332970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:55.633200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329796+0000","last_change":"2024-04-24T23:11:01.210803+0000","last_active":"2024-04-24T23:11:49.329796+0000","last_peered":"2024-04-24T23:11:49.329796+0000","last_clean":"2024-04-24T23:11:49.329796+0000","last_became_active":"2024-04-24T23:11:01.210154+0000","last_became_peered":"2024-04-24T23:11:01.210154+0000","last_unstale":"2024-04-24T23:11:49.329796+0000","last_undegraded":"2024-04-24T23:11:49.329796+0000","last_fullsized":"2024-04-24T23:11:49.329796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:08.106252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326237+0000","last_change":"2024-04-24T23:11:00.305518+0000","last_active":"2024-04-24T23:11:49.326237+0000","last_peered":"2024-04-24T23:11:49.326237+0000","last_clean":"2024-04-24T23:11:49.326237+0000","last_became_active":"2024-04-24T23:11:00.305192+0000","last_became_peered":"2024-04-24T23:11:00.305192+0000","last_unstale":"2024-04-24T23:11:49.326237+0000","last_undegraded":"2024-04-24T23:11:49.326237+0000","last_fullsized":"2024-04-24T23:11:49.326237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:43:36.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332438+0000","last_change":"2024-04-24T23:11:01.207723+0000","last_active":"2024-04-24T23:11:49.332438+0000","last_peered":"2024-04-24T23:11:49.332438+0000","last_clean":"2024-04-24T23:11:49.332438+0000","last_became_active":"2024-04-24T23:11:01.207431+0000","last_became_peered":"2024-04-24T23:11:01.207431+0000","last_unstale":"2024-04-24T23:11:49.332438+0000","last_undegraded":"2024-04-24T23:11:49.332438+0000","last_fullsized":"2024-04-24T23:11:49.332438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:06.883404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329515+0000","last_change":"2024-04-24T23:11:00.310700+0000","last_active":"2024-04-24T23:11:49.329515+0000","last_peered":"2024-04-24T23:11:49.329515+0000","last_clean":"2024-04-24T23:11:49.329515+0000","last_became_active":"2024-04-24T23:11:00.310216+0000","last_became_peered":"2024-04-24T23:11:00.310216+0000","last_unstale":"2024-04-24T23:11:49.329515+0000","last_undegraded":"2024-04-24T23:11:49.329515+0000","last_fullsized":"2024-04-24T23:11:49.329515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:25.326695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332744+0000","last_change":"2024-04-24T23:11:01.214385+0000","last_active":"2024-04-24T23:11:49.332744+0000","last_peered":"2024-04-24T23:11:49.332744+0000","last_clean":"2024-04-24T23:11:49.332744+0000","last_became_active":"2024-04-24T23:11:01.214090+0000","last_became_peered":"2024-04-24T23:11:01.214090+0000","last_unstale":"2024-04-24T23:11:49.332744+0000","last_undegraded":"2024-04-24T23:11:49.332744+0000","last_fullsized":"2024-04-24T23:11:49.332744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:57.024833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.711776+0000","last_change":"2024-04-24T23:11:49.711351+0000","last_active":"2024-04-24T23:11:49.711776+0000","last_peered":"2024-04-24T23:11:49.711776+0000","last_clean":"2024-04-24T23:11:49.711776+0000","last_became_active":"2024-04-24T23:11:49.710985+0000","last_became_peered":"2024-04-24T23:11:49.710985+0000","last_unstale":"2024-04-24T23:11:49.711776+0000","last_undegraded":"2024-04-24T23:11:49.711776+0000","last_fullsized":"2024-04-24T23:11:49.711776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:42.064044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:12:15.988342+0000","last_change":"2024-04-24T23:11:00.305349+0000","last_active":"2024-04-24T23:12:15.988342+0000","last_peered":"2024-04-24T23:12:15.988342+0000","last_clean":"2024-04-24T23:12:15.988342+0000","last_became_active":"2024-04-24T23:11:00.305046+0000","last_became_peered":"2024-04-24T23:11:00.305046+0000","last_unstale":"2024-04-24T23:12:15.988342+0000","last_undegraded":"2024-04-24T23:12:15.988342+0000","last_fullsized":"2024-04-24T23:12:15.988342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:56.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331654+0000","last_change":"2024-04-24T23:11:01.216209+0000","last_active":"2024-04-24T23:11:49.331654+0000","last_peered":"2024-04-24T23:11:49.331654+0000","last_clean":"2024-04-24T23:11:49.331654+0000","last_became_active":"2024-04-24T23:11:01.215899+0000","last_became_peered":"2024-04-24T23:11:01.215899+0000","last_unstale":"2024-04-24T23:11:49.331654+0000","last_undegraded":"2024-04-24T23:11:49.331654+0000","last_fullsized":"2024-04-24T23:11:49.331654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:09.404193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.711060+0000","last_change":"2024-04-24T23:11:00.303160+0000","last_active":"2024-04-24T23:11:49.711060+0000","last_peered":"2024-04-24T23:11:49.711060+0000","last_clean":"2024-04-24T23:11:49.711060+0000","last_became_active":"2024-04-24T23:11:00.302781+0000","last_became_peered":"2024-04-24T23:11:00.302781+0000","last_unstale":"2024-04-24T23:11:49.711060+0000","last_undegraded":"2024-04-24T23:11:49.711060+0000","last_fullsized":"2024-04-24T23:11:49.711060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:42.340993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329413+0000","last_change":"2024-04-24T23:09:43.738402+0000","last_active":"2024-04-24T23:11:49.329413+0000","last_peered":"2024-04-24T23:11:49.329413+0000","last_clean":"2024-04-24T23:11:49.329413+0000","last_became_active":"2024-04-24T23:09:43.737979+0000","last_became_peered":"2024-04-24T23:09:43.737979+0000","last_unstale":"2024-04-24T23:11:49.329413+0000","last_undegraded":"2024-04-24T23:11:49.329413+0000","last_fullsized":"2024-04-24T23:11:49.329413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:06:16.456818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:06:16.456818+0000","last_clean_scrub_stamp":"2024-04-24T23:06:16.456818+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:52.449055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334022+0000","last_change":"2024-04-24T23:11:01.203542+0000","last_active":"2024-04-24T23:11:49.334022+0000","last_peered":"2024-04-24T23:11:49.334022+0000","last_clean":"2024-04-24T23:11:49.334022+0000","last_became_active":"2024-04-24T23:11:01.203111+0000","last_became_peered":"2024-04-24T23:11:01.203111+0000","last_unstale":"2024-04-24T23:11:49.334022+0000","last_undegraded":"2024-04-24T23:11:49.334022+0000","last_fullsized":"2024-04-24T23:11:49.334022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:27.833165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329511+0000","last_change":"2024-04-24T23:11:00.304542+0000","last_active":"2024-04-24T23:11:49.329511+0000","last_peered":"2024-04-24T23:11:49.329511+0000","last_clean":"2024-04-24T23:11:49.329511+0000","last_became_active":"2024-04-24T23:11:00.304234+0000","last_became_peered":"2024-04-24T23:11:00.304234+0000","last_unstale":"2024-04-24T23:11:49.329511+0000","last_undegraded":"2024-04-24T23:11:49.329511+0000","last_fullsized":"2024-04-24T23:11:49.329511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:18.318876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333377+0000","last_change":"2024-04-24T23:11:00.310962+0000","last_active":"2024-04-24T23:11:49.333377+0000","last_peered":"2024-04-24T23:11:49.333377+0000","last_clean":"2024-04-24T23:11:49.333377+0000","last_became_active":"2024-04-24T23:11:00.310648+0000","last_became_peered":"2024-04-24T23:11:00.310648+0000","last_unstale":"2024-04-24T23:11:49.333377+0000","last_undegraded":"2024-04-24T23:11:49.333377+0000","last_fullsized":"2024-04-24T23:11:49.333377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:01.693517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334835+0000","last_change":"2024-04-24T23:11:01.200293+0000","last_active":"2024-04-24T23:11:49.334835+0000","last_peered":"2024-04-24T23:11:49.334835+0000","last_clean":"2024-04-24T23:11:49.334835+0000","last_became_active":"2024-04-24T23:11:01.199945+0000","last_became_peered":"2024-04-24T23:11:01.199945+0000","last_unstale":"2024-04-24T23:11:49.334835+0000","last_undegraded":"2024-04-24T23:11:49.334835+0000","last_fullsized":"2024-04-24T23:11:49.334835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:25.258253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.326436+0000","last_change":"2024-04-24T23:11:00.307467+0000","last_active":"2024-04-24T23:11:49.326436+0000","last_peered":"2024-04-24T23:11:49.326436+0000","last_clean":"2024-04-24T23:11:49.326436+0000","last_became_active":"2024-04-24T23:11:00.307147+0000","last_became_peered":"2024-04-24T23:11:00.307147+0000","last_unstale":"2024-04-24T23:11:49.326436+0000","last_undegraded":"2024-04-24T23:11:49.326436+0000","last_fullsized":"2024-04-24T23:11:49.326436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:42.955410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331886+0000","last_change":"2024-04-24T23:11:01.210541+0000","last_active":"2024-04-24T23:11:49.331886+0000","last_peered":"2024-04-24T23:11:49.331886+0000","last_clean":"2024-04-24T23:11:49.331886+0000","last_became_active":"2024-04-24T23:11:01.210187+0000","last_became_peered":"2024-04-24T23:11:01.210187+0000","last_unstale":"2024-04-24T23:11:49.331886+0000","last_undegraded":"2024-04-24T23:11:49.331886+0000","last_fullsized":"2024-04-24T23:11:49.331886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:45.138001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.711027+0000","last_change":"2024-04-24T23:11:00.308782+0000","last_active":"2024-04-24T23:11:49.711027+0000","last_peered":"2024-04-24T23:11:49.711027+0000","last_clean":"2024-04-24T23:11:49.711027+0000","last_became_active":"2024-04-24T23:11:00.308269+0000","last_became_peered":"2024-04-24T23:11:00.308269+0000","last_unstale":"2024-04-24T23:11:49.711027+0000","last_undegraded":"2024-04-24T23:11:49.711027+0000","last_fullsized":"2024-04-24T23:11:49.711027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:05.089037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335979+0000","last_change":"2024-04-24T23:11:49.335979+0000","last_active":"2024-04-24T23:11:49.335979+0000","last_peered":"2024-04-24T23:11:49.335979+0000","last_clean":"2024-04-24T23:11:49.335979+0000","last_became_active":"2024-04-24T23:11:49.335562+0000","last_became_peered":"2024-04-24T23:11:49.335562+0000","last_unstale":"2024-04-24T23:11:49.335979+0000","last_undegraded":"2024-04-24T23:11:49.335979+0000","last_fullsized":"2024-04-24T23:11:49.335979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:45.810023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335666+0000","last_change":"2024-04-24T23:11:00.307522+0000","last_active":"2024-04-24T23:11:49.335666+0000","last_peered":"2024-04-24T23:11:49.335666+0000","last_clean":"2024-04-24T23:11:49.335666+0000","last_became_active":"2024-04-24T23:11:00.306106+0000","last_became_peered":"2024-04-24T23:11:00.306106+0000","last_unstale":"2024-04-24T23:11:49.335666+0000","last_undegraded":"2024-04-24T23:11:49.335666+0000","last_fullsized":"2024-04-24T23:11:49.335666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:21.623996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332269+0000","last_change":"2024-04-24T23:11:01.207426+0000","last_active":"2024-04-24T23:11:49.332269+0000","last_peered":"2024-04-24T23:11:49.332269+0000","last_clean":"2024-04-24T23:11:49.332269+0000","last_became_active":"2024-04-24T23:11:01.207020+0000","last_became_peered":"2024-04-24T23:11:01.207020+0000","last_unstale":"2024-04-24T23:11:49.332269+0000","last_undegraded":"2024-04-24T23:11:49.332269+0000","last_fullsized":"2024-04-24T23:11:49.332269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:55:58.389998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331365+0000","last_change":"2024-04-24T23:11:00.302686+0000","last_active":"2024-04-24T23:11:49.331365+0000","last_peered":"2024-04-24T23:11:49.331365+0000","last_clean":"2024-04-24T23:11:49.331365+0000","last_became_active":"2024-04-24T23:11:00.302362+0000","last_became_peered":"2024-04-24T23:11:00.302362+0000","last_unstale":"2024-04-24T23:11:49.331365+0000","last_undegraded":"2024-04-24T23:11:49.331365+0000","last_fullsized":"2024-04-24T23:11:49.331365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:30.402780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329504+0000","last_change":"2024-04-24T23:11:01.212413+0000","last_active":"2024-04-24T23:11:49.329504+0000","last_peered":"2024-04-24T23:11:49.329504+0000","last_clean":"2024-04-24T23:11:49.329504+0000","last_became_active":"2024-04-24T23:11:01.211975+0000","last_became_peered":"2024-04-24T23:11:01.211975+0000","last_unstale":"2024-04-24T23:11:49.329504+0000","last_undegraded":"2024-04-24T23:11:49.329504+0000","last_fullsized":"2024-04-24T23:11:49.329504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:56.129977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.339617+0000","last_change":"2024-04-24T23:11:49.339214+0000","last_active":"2024-04-24T23:11:49.339617+0000","last_peered":"2024-04-24T23:11:49.339617+0000","last_clean":"2024-04-24T23:11:49.339617+0000","last_became_active":"2024-04-24T23:11:49.334513+0000","last_became_peered":"2024-04-24T23:11:49.334513+0000","last_unstale":"2024-04-24T23:11:49.339617+0000","last_undegraded":"2024-04-24T23:11:49.339617+0000","last_fullsized":"2024-04-24T23:11:49.339617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:21.817319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331454+0000","last_change":"2024-04-24T23:11:01.214041+0000","last_active":"2024-04-24T23:11:49.331454+0000","last_peered":"2024-04-24T23:11:49.331454+0000","last_clean":"2024-04-24T23:11:49.331454+0000","last_became_active":"2024-04-24T23:11:01.213639+0000","last_became_peered":"2024-04-24T23:11:01.213639+0000","last_unstale":"2024-04-24T23:11:49.331454+0000","last_undegraded":"2024-04-24T23:11:49.331454+0000","last_fullsized":"2024-04-24T23:11:49.331454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:08.819111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333807+0000","last_change":"2024-04-24T23:11:00.303790+0000","last_active":"2024-04-24T23:11:49.333807+0000","last_peered":"2024-04-24T23:11:49.333807+0000","last_clean":"2024-04-24T23:11:49.333807+0000","last_became_active":"2024-04-24T23:11:00.303381+0000","last_became_peered":"2024-04-24T23:11:00.303381+0000","last_unstale":"2024-04-24T23:11:49.333807+0000","last_undegraded":"2024-04-24T23:11:49.333807+0000","last_fullsized":"2024-04-24T23:11:49.333807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:18.725608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329573+0000","last_change":"2024-04-24T23:11:01.219575+0000","last_active":"2024-04-24T23:11:49.329573+0000","last_peered":"2024-04-24T23:11:49.329573+0000","last_clean":"2024-04-24T23:11:49.329573+0000","last_became_active":"2024-04-24T23:11:01.218750+0000","last_became_peered":"2024-04-24T23:11:01.218750+0000","last_unstale":"2024-04-24T23:11:49.329573+0000","last_undegraded":"2024-04-24T23:11:49.329573+0000","last_fullsized":"2024-04-24T23:11:49.329573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:00.945493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.333162+0000","last_change":"2024-04-24T23:11:00.310468+0000","last_active":"2024-04-24T23:11:49.333162+0000","last_peered":"2024-04-24T23:11:49.333162+0000","last_clean":"2024-04-24T23:11:49.333162+0000","last_became_active":"2024-04-24T23:11:00.310176+0000","last_became_peered":"2024-04-24T23:11:00.310176+0000","last_unstale":"2024-04-24T23:11:49.333162+0000","last_undegraded":"2024-04-24T23:11:49.333162+0000","last_fullsized":"2024-04-24T23:11:49.333162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:25.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329819+0000","last_change":"2024-04-24T23:11:01.219593+0000","last_active":"2024-04-24T23:11:49.329819+0000","last_peered":"2024-04-24T23:11:49.329819+0000","last_clean":"2024-04-24T23:11:49.329819+0000","last_became_active":"2024-04-24T23:11:01.219247+0000","last_became_peered":"2024-04-24T23:11:01.219247+0000","last_unstale":"2024-04-24T23:11:49.329819+0000","last_undegraded":"2024-04-24T23:11:49.329819+0000","last_fullsized":"2024-04-24T23:11:49.329819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:12.605797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335957+0000","last_change":"2024-04-24T23:11:00.305124+0000","last_active":"2024-04-24T23:11:49.335957+0000","last_peered":"2024-04-24T23:11:49.335957+0000","last_clean":"2024-04-24T23:11:49.335957+0000","last_became_active":"2024-04-24T23:11:00.301468+0000","last_became_peered":"2024-04-24T23:11:00.301468+0000","last_unstale":"2024-04-24T23:11:49.335957+0000","last_undegraded":"2024-04-24T23:11:49.335957+0000","last_fullsized":"2024-04-24T23:11:49.335957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:58.451956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329501+0000","last_change":"2024-04-24T23:11:01.209967+0000","last_active":"2024-04-24T23:11:49.329501+0000","last_peered":"2024-04-24T23:11:49.329501+0000","last_clean":"2024-04-24T23:11:49.329501+0000","last_became_active":"2024-04-24T23:11:01.209607+0000","last_became_peered":"2024-04-24T23:11:01.209607+0000","last_unstale":"2024-04-24T23:11:49.329501+0000","last_undegraded":"2024-04-24T23:11:49.329501+0000","last_fullsized":"2024-04-24T23:11:49.329501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:02.840211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334745+0000","last_change":"2024-04-24T23:11:00.308200+0000","last_active":"2024-04-24T23:11:49.334745+0000","last_peered":"2024-04-24T23:11:49.334745+0000","last_clean":"2024-04-24T23:11:49.334745+0000","last_became_active":"2024-04-24T23:11:00.306369+0000","last_became_peered":"2024-04-24T23:11:00.306369+0000","last_unstale":"2024-04-24T23:11:49.334745+0000","last_undegraded":"2024-04-24T23:11:49.334745+0000","last_fullsized":"2024-04-24T23:11:49.334745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:10:59.281124+0000","last_clean_scrub_stamp":"2024-04-24T23:10:59.281124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:24.462361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329883+0000","last_change":"2024-04-24T23:11:01.207706+0000","last_active":"2024-04-24T23:11:49.329883+0000","last_peered":"2024-04-24T23:11:49.329883+0000","last_clean":"2024-04-24T23:11:49.329883+0000","last_became_active":"2024-04-24T23:11:01.207357+0000","last_became_peered":"2024-04-24T23:11:01.207357+0000","last_unstale":"2024-04-24T23:11:49.329883+0000","last_undegraded":"2024-04-24T23:11:49.329883+0000","last_fullsized":"2024-04-24T23:11:49.329883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:46.785844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329606+0000","last_change":"2024-04-24T23:11:01.214886+0000","last_active":"2024-04-24T23:11:49.329606+0000","last_peered":"2024-04-24T23:11:49.329606+0000","last_clean":"2024-04-24T23:11:49.329606+0000","last_became_active":"2024-04-24T23:11:01.213671+0000","last_became_peered":"2024-04-24T23:11:01.213671+0000","last_unstale":"2024-04-24T23:11:49.329606+0000","last_undegraded":"2024-04-24T23:11:49.329606+0000","last_fullsized":"2024-04-24T23:11:49.329606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:44.777437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332752+0000","last_change":"2024-04-24T23:11:01.203578+0000","last_active":"2024-04-24T23:11:49.332752+0000","last_peered":"2024-04-24T23:11:49.332752+0000","last_clean":"2024-04-24T23:11:49.332752+0000","last_became_active":"2024-04-24T23:11:01.203250+0000","last_became_peered":"2024-04-24T23:11:01.203250+0000","last_unstale":"2024-04-24T23:11:49.332752+0000","last_undegraded":"2024-04-24T23:11:49.332752+0000","last_fullsized":"2024-04-24T23:11:49.332752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:39.050149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332243+0000","last_change":"2024-04-24T23:11:01.209641+0000","last_active":"2024-04-24T23:11:49.332243+0000","last_peered":"2024-04-24T23:11:49.332243+0000","last_clean":"2024-04-24T23:11:49.332243+0000","last_became_active":"2024-04-24T23:11:01.209110+0000","last_became_peered":"2024-04-24T23:11:01.209110+0000","last_unstale":"2024-04-24T23:11:49.332243+0000","last_undegraded":"2024-04-24T23:11:49.332243+0000","last_fullsized":"2024-04-24T23:11:49.332243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:14.709926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.328727+0000","last_change":"2024-04-24T23:11:01.213894+0000","last_active":"2024-04-24T23:11:49.328727+0000","last_peered":"2024-04-24T23:11:49.328727+0000","last_clean":"2024-04-24T23:11:49.328727+0000","last_became_active":"2024-04-24T23:11:01.213461+0000","last_became_peered":"2024-04-24T23:11:01.213461+0000","last_unstale":"2024-04-24T23:11:49.328727+0000","last_undegraded":"2024-04-24T23:11:49.328727+0000","last_fullsized":"2024-04-24T23:11:49.328727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:00.786413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334742+0000","last_change":"2024-04-24T23:11:01.206556+0000","last_active":"2024-04-24T23:11:49.334742+0000","last_peered":"2024-04-24T23:11:49.334742+0000","last_clean":"2024-04-24T23:11:49.334742+0000","last_became_active":"2024-04-24T23:11:01.206178+0000","last_became_peered":"2024-04-24T23:11:01.206178+0000","last_unstale":"2024-04-24T23:11:49.334742+0000","last_undegraded":"2024-04-24T23:11:49.334742+0000","last_fullsized":"2024-04-24T23:11:49.334742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:56.765847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329846+0000","last_change":"2024-04-24T23:11:01.213174+0000","last_active":"2024-04-24T23:11:49.329846+0000","last_peered":"2024-04-24T23:11:49.329846+0000","last_clean":"2024-04-24T23:11:49.329846+0000","last_became_active":"2024-04-24T23:11:01.212815+0000","last_became_peered":"2024-04-24T23:11:01.212815+0000","last_unstale":"2024-04-24T23:11:49.329846+0000","last_undegraded":"2024-04-24T23:11:49.329846+0000","last_fullsized":"2024-04-24T23:11:49.329846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:09.281324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.332745+0000","last_change":"2024-04-24T23:11:01.215030+0000","last_active":"2024-04-24T23:11:49.332745+0000","last_peered":"2024-04-24T23:11:49.332745+0000","last_clean":"2024-04-24T23:11:49.332745+0000","last_became_active":"2024-04-24T23:11:01.214386+0000","last_became_peered":"2024-04-24T23:11:01.214386+0000","last_unstale":"2024-04-24T23:11:49.332745+0000","last_undegraded":"2024-04-24T23:11:49.332745+0000","last_fullsized":"2024-04-24T23:11:49.332745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:10.633565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331839+0000","last_change":"2024-04-24T23:11:01.217748+0000","last_active":"2024-04-24T23:11:49.331839+0000","last_peered":"2024-04-24T23:11:49.331839+0000","last_clean":"2024-04-24T23:11:49.331839+0000","last_became_active":"2024-04-24T23:11:01.217403+0000","last_became_peered":"2024-04-24T23:11:01.217403+0000","last_unstale":"2024-04-24T23:11:49.331839+0000","last_undegraded":"2024-04-24T23:11:49.331839+0000","last_fullsized":"2024-04-24T23:11:49.331839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:46:29.569179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335433+0000","last_change":"2024-04-24T23:11:01.208894+0000","last_active":"2024-04-24T23:11:49.335433+0000","last_peered":"2024-04-24T23:11:49.335433+0000","last_clean":"2024-04-24T23:11:49.335433+0000","last_became_active":"2024-04-24T23:11:01.208576+0000","last_became_peered":"2024-04-24T23:11:01.208576+0000","last_unstale":"2024-04-24T23:11:49.335433+0000","last_undegraded":"2024-04-24T23:11:49.335433+0000","last_fullsized":"2024-04-24T23:11:49.335433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:13.967021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.331335+0000","last_change":"2024-04-24T23:11:01.218561+0000","last_active":"2024-04-24T23:11:49.331335+0000","last_peered":"2024-04-24T23:11:49.331335+0000","last_clean":"2024-04-24T23:11:49.331335+0000","last_became_active":"2024-04-24T23:11:01.218261+0000","last_became_peered":"2024-04-24T23:11:01.218261+0000","last_unstale":"2024-04-24T23:11:49.331335+0000","last_undegraded":"2024-04-24T23:11:49.331335+0000","last_fullsized":"2024-04-24T23:11:49.331335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:49.677225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.335228+0000","last_change":"2024-04-24T23:11:01.214868+0000","last_active":"2024-04-24T23:11:49.335228+0000","last_peered":"2024-04-24T23:11:49.335228+0000","last_clean":"2024-04-24T23:11:49.335228+0000","last_became_active":"2024-04-24T23:11:01.214582+0000","last_became_peered":"2024-04-24T23:11:01.214582+0000","last_unstale":"2024-04-24T23:11:49.335228+0000","last_undegraded":"2024-04-24T23:11:49.335228+0000","last_fullsized":"2024-04-24T23:11:49.335228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:48.963896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.334444+0000","last_change":"2024-04-24T23:11:01.212348+0000","last_active":"2024-04-24T23:11:49.334444+0000","last_peered":"2024-04-24T23:11:49.334444+0000","last_clean":"2024-04-24T23:11:49.334444+0000","last_became_active":"2024-04-24T23:11:01.211802+0000","last_became_peered":"2024-04-24T23:11:01.211802+0000","last_unstale":"2024-04-24T23:11:49.334444+0000","last_undegraded":"2024-04-24T23:11:49.334444+0000","last_fullsized":"2024-04-24T23:11:49.334444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:02:49.484245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T23:11:49.329694+0000","last_change":"2024-04-24T23:11:01.209834+0000","last_active":"2024-04-24T23:11:49.329694+0000","last_peered":"2024-04-24T23:11:49.329694+0000","last_clean":"2024-04-24T23:11:49.329694+0000","last_became_active":"2024-04-24T23:11:01.209476+0000","last_became_peered":"2024-04-24T23:11:01.209476+0000","last_unstale":"2024-04-24T23:11:49.329694+0000","last_undegraded":"2024-04-24T23:11:49.329694+0000","last_fullsized":"2024-04-24T23:11:49.329694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:11:00.283142+0000","last_clean_scrub_stamp":"2024-04-24T23:11:00.283142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:20.036684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244416,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63304,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":455,"num_read_kb":294,"num_write":154,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":58912,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":189912,"data_compressed":13352,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":460802,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1111586,"data_compressed":43319,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":461038,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":460958,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1053279,"data_compressed":41436,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":460427,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":519132,"data_compressed":7700,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":461698,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":461961,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":460738,"data_compressed":5817,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":520167,"data_compressed":15403,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":1052572,"data_compressed":41436,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33980,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":58912,"data_compressed":1883,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":59947,"data_compressed":9586,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60881,"data_compressed":1883,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:12:20.777 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T23:12:20.777 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T23:12:20.777 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T23:12:20.777 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T23:12:21.016 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:21.439 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:21 smithi102 ceph-mon[29216]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:21.439 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:21 smithi102 ceph-mon[29216]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:21.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:21 smithi161 ceph-mon[32709]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:21.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:21 smithi161 ceph-mon[32709]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:21.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:21 smithi183 ceph-mon[33137]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:21.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:21 smithi183 ceph-mon[33137]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:22.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.815+0000 7f7a33b19700 1 -- 172.21.15.102:0/3621689534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 msgr2=0x7f7a2c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:22.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.815+0000 7f7a33b19700 1 --2- 172.21.15.102:0/3621689534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 0x7f7a2c06a320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009990 tx=0x7f7a28009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.816+0000 7f7a33b19700 1 -- 172.21.15.102:0/3621689534 shutdown_connections 2024-04-24T23:12:22.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.816+0000 7f7a33b19700 1 --2- 172.21.15.102:0/3621689534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a2c10ece0 0x7f7a2c1110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.816+0000 7f7a33b19700 1 --2- 172.21.15.102:0/3621689534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 0x7f7a2c06a320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.816+0000 7f7a33b19700 1 --2- 172.21.15.102:0/3621689534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a2c06a860 0x7f7a2c10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.816+0000 7f7a33b19700 1 -- 172.21.15.102:0/3621689534 >> 172.21.15.102:0/3621689534 conn(0x7f7a2c0780e0 msgr2=0x7f7a2c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:22.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.816+0000 7f7a33b19700 1 -- 172.21.15.102:0/3621689534 shutdown_connections 2024-04-24T23:12:22.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.816+0000 7f7a33b19700 1 -- 172.21.15.102:0/3621689534 wait complete. 2024-04-24T23:12:22.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a33b19700 1 Processor -- start 2024-04-24T23:12:22.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a33b19700 1 -- start start 2024-04-24T23:12:22.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a33b19700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 0x7f7a2c1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:22.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a33b19700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a2c06a860 0x7f7a2c1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a33b19700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a2c10ece0 0x7f7a2c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a33b19700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1145f0 con 0x7f7a2c069f40 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a33b19700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c114470 con 0x7f7a2c06a860 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a33b19700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c114770 con 0x7f7a2c10ece0 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a318b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 0x7f7a2c1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a318b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 0x7f7a2c1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60746/0 (socket says 172.21.15.102:60746) 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a318b5700 1 -- 172.21.15.102:0/1968794115 learned_addr learned my addr 172.21.15.102:0/1968794115 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a310b4700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a2c06a860 0x7f7a2c1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a320b6700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a2c10ece0 0x7f7a2c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a318b5700 1 -- 172.21.15.102:0/1968794115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a2c10ece0 msgr2=0x7f7a2c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a318b5700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a2c10ece0 0x7f7a2c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a318b5700 1 -- 172.21.15.102:0/1968794115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a2c06a860 msgr2=0x7f7a2c1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a318b5700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a2c06a860 0x7f7a2c1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:22.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a318b5700 1 -- 172.21.15.102:0/1968794115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c19b0b0 con 0x7f7a2c069f40 2024-04-24T23:12:22.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a320b6700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a2c10ece0 0x7f7a2c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:22.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.817+0000 7f7a310b4700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a2c06a860 0x7f7a2c1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:22.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.818+0000 7f7a318b5700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 0x7f7a2c1a0740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7a280057c0 tx=0x7f7a280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:22.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.818+0000 7f7a22ffd700 1 -- 172.21.15.102:0/1968794115 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28027070 con 0x7f7a2c069f40 2024-04-24T23:12:22.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.818+0000 7f7a22ffd700 1 -- 172.21.15.102:0/1968794115 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a28022470 con 0x7f7a2c069f40 2024-04-24T23:12:22.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.818+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c19b340 con 0x7f7a2c069f40 2024-04-24T23:12:22.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.818+0000 7f7a22ffd700 1 -- 172.21.15.102:0/1968794115 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28004230 con 0x7f7a2c069f40 2024-04-24T23:12:22.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.818+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c1a7610 con 0x7f7a2c069f40 2024-04-24T23:12:22.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.819+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c068f20 con 0x7f7a2c069f40 2024-04-24T23:12:22.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.820+0000 7f7a22ffd700 1 -- 172.21.15.102:0/1968794115 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7a2802a070 con 0x7f7a2c069f40 2024-04-24T23:12:22.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.821+0000 7f7a22ffd700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7a180a2c20 0x7f7a180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:22.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.821+0000 7f7a310b4700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7a180a2c20 0x7f7a180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:22.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.821+0000 7f7a22ffd700 1 -- 172.21.15.102:0/1968794115 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7a280cfed0 con 0x7f7a2c069f40 2024-04-24T23:12:22.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.821+0000 7f7a310b4700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7a180a2c20 0x7f7a180a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c005fd0 tx=0x7f7a1c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:22.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:22.825+0000 7f7a22ffd700 1 -- 172.21.15.102:0/1968794115 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7a28013070 con 0x7f7a2c069f40 2024-04-24T23:12:23.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.067+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7a2c02db30 con 0x7f7a2c069f40 2024-04-24T23:12:23.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.068+0000 7f7a22ffd700 1 -- 172.21.15.102:0/1968794115 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7a2809cfd0 con 0x7f7a2c069f40 2024-04-24T23:12:23.070 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:23.070 INFO:teuthology.orchestra.run.smithi102.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T23:12:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7a180a2c20 msgr2=0x7f7a180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7a180a2c20 0x7f7a180a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c005fd0 tx=0x7f7a1c005e30 comp rx=0 tx=0).stop 2024-04-24T23:12:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 msgr2=0x7f7a2c1a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 0x7f7a2c1a0740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7a280057c0 tx=0x7f7a280057f0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 shutdown_connections 2024-04-24T23:12:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a2c10ece0 0x7f7a2c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7a180a2c20 0x7f7a180a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a2c069f40 0x7f7a2c1a0740 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 --2- 172.21.15.102:0/1968794115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a2c06a860 0x7f7a2c1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 >> 172.21.15.102:0/1968794115 conn(0x7f7a2c0780e0 msgr2=0x7f7a2c1101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:23.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 shutdown_connections 2024-04-24T23:12:23.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.070+0000 7f7a33b19700 1 -- 172.21.15.102:0/1968794115 wait complete. 2024-04-24T23:12:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:23 smithi161 ceph-mon[32709]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-24T23:12:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:23 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1968794115' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:12:23.754 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:23 smithi102 ceph-mon[29216]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-24T23:12:23.754 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:23 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1968794115' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:12:23.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:23 smithi183 ceph-mon[33137]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-24T23:12:23.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:23 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1968794115' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:12:23.781 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T23:12:23.782 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:23.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 -- 172.21.15.102:0/1247846499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc1022a0 msgr2=0x7f79bc104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 --2- 172.21.15.102:0/1247846499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc1022a0 0x7f79bc104690 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009900 tx=0x7f79b802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:23.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 -- 172.21.15.102:0/1247846499 shutdown_connections 2024-04-24T23:12:23.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 --2- 172.21.15.102:0/1247846499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79bc107500 0x7f79bc1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 --2- 172.21.15.102:0/1247846499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc104bd0 0x7f79bc106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 --2- 172.21.15.102:0/1247846499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc1022a0 0x7f79bc104690 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 -- 172.21.15.102:0/1247846499 >> 172.21.15.102:0/1247846499 conn(0x7f79bc0fbbf0 msgr2=0x7f79bc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 -- 172.21.15.102:0/1247846499 shutdown_connections 2024-04-24T23:12:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 -- 172.21.15.102:0/1247846499 wait complete. 2024-04-24T23:12:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 Processor -- start 2024-04-24T23:12:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.883+0000 7f79c3729640 1 -- start start 2024-04-24T23:12:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c3729640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc1022a0 0x7f79bc101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:23.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c3729640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79bc104bd0 0x7f79bc100250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:23.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c3729640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc107500 0x7f79bc100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:23.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c3729640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc10d2a0 con 0x7f79bc107500 2024-04-24T23:12:23.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c3729640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc10d120 con 0x7f79bc104bd0 2024-04-24T23:12:23.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c3729640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc10d420 con 0x7f79bc1022a0 2024-04-24T23:12:23.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c1cc6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc107500 0x7f79bc100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:23.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c1cc6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc107500 0x7f79bc100790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60764/0 (socket says 172.21.15.102:60764) 2024-04-24T23:12:23.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c1cc6640 1 -- 172.21.15.102:0/2840779414 learned_addr learned my addr 172.21.15.102:0/2840779414 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:23.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c14c5640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc1022a0 0x7f79bc101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:23.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c0cc4640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79bc104bd0 0x7f79bc100250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:23.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c1cc6640 1 -- 172.21.15.102:0/2840779414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc1022a0 msgr2=0x7f79bc101c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c1cc6640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc1022a0 0x7f79bc101c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c1cc6640 1 -- 172.21.15.102:0/2840779414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79bc104bd0 msgr2=0x7f79bc100250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c1cc6640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79bc104bd0 0x7f79bc100250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c1cc6640 1 -- 172.21.15.102:0/2840779414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79bc101020 con 0x7f79bc107500 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c14c5640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc1022a0 0x7f79bc101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c0cc4640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79bc104bd0 0x7f79bc100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c1cc6640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc107500 0x7f79bc100790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009fd0 tx=0x7f79b400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79b27fc640 1 -- 172.21.15.102:0/2840779414 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4017070 con 0x7f79bc107500 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79b27fc640 1 -- 172.21.15.102:0/2840779414 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79b4003e20 con 0x7f79bc107500 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79b27fc640 1 -- 172.21.15.102:0/2840779414 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40106c0 con 0x7f79bc107500 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79bc1a1b30 con 0x7f79bc107500 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.884+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79bc1a2080 con 0x7f79bc107500 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.885+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79bc0fd6e0 con 0x7f79bc107500 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.886+0000 7f79b27fc640 1 -- 172.21.15.102:0/2840779414 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f79b4003970 con 0x7f79bc107500 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.887+0000 7f79b27fc640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f79a80a2c10 0x7f79a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:23.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.887+0000 7f79c14c5640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f79a80a2c10 0x7f79a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:23.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.887+0000 7f79b27fc640 1 -- 172.21.15.102:0/2840779414 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f79b40cede0 con 0x7f79bc107500 2024-04-24T23:12:23.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.888+0000 7f79c14c5640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f79a80a2c10 0x7f79a80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f79b80314e0 tx=0x7f79b8032c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:23.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:23.891+0000 7f79b27fc640 1 -- 172.21.15.102:0/2840779414 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f79b409bcd0 con 0x7f79bc107500 2024-04-24T23:12:24.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.075+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79bc0623e0 con 0x7f79bc107500 2024-04-24T23:12:24.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.077+0000 7f79b27fc640 1 -- 172.21.15.102:0/2840779414 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22720 (secure 0 0 0) 0x7f79b40047e0 con 0x7f79bc107500 2024-04-24T23:12:24.079 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:24.079 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:11:49.323161+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f79a80a2c10 msgr2=0x7f79a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f79a80a2c10 0x7f79a80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f79b80314e0 tx=0x7f79b8032c60 comp rx=0 tx=0).stop 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc107500 msgr2=0x7f79bc100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc107500 0x7f79bc100790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009fd0 tx=0x7f79b400b260 comp rx=0 tx=0).stop 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 shutdown_connections 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f79a80a2c10 0x7f79a80a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79bc107500 0x7f79bc100790 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79bc104bd0 0x7f79bc100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 --2- 172.21.15.102:0/2840779414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc1022a0 0x7f79bc101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 >> 172.21.15.102:0/2840779414 conn(0x7f79bc0fbbf0 msgr2=0x7f79bc102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 shutdown_connections 2024-04-24T23:12:24.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.079+0000 7f79c3729640 1 -- 172.21.15.102:0/2840779414 wait complete. 2024-04-24T23:12:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:58.730586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:59.855279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:12:24.093 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:24.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 -- 172.21.15.102:0/4170456204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698102200 msgr2=0x7fb6981045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 --2- 172.21.15.102:0/4170456204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698102200 0x7fb6981045f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009900 tx=0x7fb68c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:24.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 -- 172.21.15.102:0/4170456204 shutdown_connections 2024-04-24T23:12:24.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 --2- 172.21.15.102:0/4170456204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb698107460 0x7fb6981098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 --2- 172.21.15.102:0/4170456204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698104b30 0x7fb698106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 --2- 172.21.15.102:0/4170456204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698102200 0x7fb6981045f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 -- 172.21.15.102:0/4170456204 >> 172.21.15.102:0/4170456204 conn(0x7fb6980fbbf0 msgr2=0x7fb6980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 -- 172.21.15.102:0/4170456204 shutdown_connections 2024-04-24T23:12:24.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 -- 172.21.15.102:0/4170456204 wait complete. 2024-04-24T23:12:24.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.194+0000 7fb69e35a640 1 Processor -- start 2024-04-24T23:12:24.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb69e35a640 1 -- start start 2024-04-24T23:12:24.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb69e35a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb698102200 0x7fb698197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb69e35a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698104b30 0x7fb698197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb69e35a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698107460 0x7fb69819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb69e35a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb69810d2d0 con 0x7fb698104b30 2024-04-24T23:12:24.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb69e35a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb69810d150 con 0x7fb698107460 2024-04-24T23:12:24.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb69e35a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb69810d450 con 0x7fb698102200 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb6977fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698104b30 0x7fb698197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb6977fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698104b30 0x7fb698197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60788/0 (socket says 172.21.15.102:60788) 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb6977fe640 1 -- 172.21.15.102:0/2051320025 learned_addr learned my addr 172.21.15.102:0/2051320025 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb697fff640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb698102200 0x7fb698197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb6977fe640 1 -- 172.21.15.102:0/2051320025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb698102200 msgr2=0x7fb698197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb6977fe640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb698102200 0x7fb698197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb6977fe640 1 -- 172.21.15.102:0/2051320025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698107460 msgr2=0x7fb69819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb69c8f7640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698107460 0x7fb69819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb6977fe640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698107460 0x7fb69819f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb6977fe640 1 -- 172.21.15.102:0/2051320025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69819f670 con 0x7fb698104b30 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.195+0000 7fb6977fe640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698104b30 0x7fb698197f90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb68800c970 tx=0x7fb68800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.196+0000 7fb6957fa640 1 -- 172.21.15.102:0/2051320025 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688007bb0 con 0x7fb698104b30 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.196+0000 7fb6957fa640 1 -- 172.21.15.102:0/2051320025 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb688007d50 con 0x7fb698104b30 2024-04-24T23:12:24.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.196+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69819f960 con 0x7fb698104b30 2024-04-24T23:12:24.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.196+0000 7fb6957fa640 1 -- 172.21.15.102:0/2051320025 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68800f450 con 0x7fb698104b30 2024-04-24T23:12:24.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.196+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69819feb0 con 0x7fb698104b30 2024-04-24T23:12:24.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.197+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6980fd6e0 con 0x7fb698104b30 2024-04-24T23:12:24.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.198+0000 7fb6957fa640 1 -- 172.21.15.102:0/2051320025 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb688004f20 con 0x7fb698104b30 2024-04-24T23:12:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.198+0000 7fb6957fa640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb6800a2b90 0x7fb6800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.199+0000 7fb697fff640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb6800a2b90 0x7fb6800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.199+0000 7fb6957fa640 1 -- 172.21.15.102:0/2051320025 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb6880cf140 con 0x7fb698104b30 2024-04-24T23:12:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.199+0000 7fb697fff640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb6800a2b90 0x7fb6800a5050 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb68c000c00 tx=0x7fb68c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.203+0000 7fb6957fa640 1 -- 172.21.15.102:0/2051320025 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb688014030 con 0x7fb698104b30 2024-04-24T23:12:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.385+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb698062340 con 0x7fb698104b30 2024-04-24T23:12:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.385+0000 7fb6957fa640 1 -- 172.21.15.102:0/2051320025 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fb68809c090 con 0x7fb698104b30 2024-04-24T23:12:24.387 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:12:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb6800a2b90 msgr2=0x7fb6800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb6800a2b90 0x7fb6800a5050 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb68c000c00 tx=0x7fb68c0314a0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698104b30 msgr2=0x7fb698197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698104b30 0x7fb698197f90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb68800c970 tx=0x7fb68800ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 shutdown_connections 2024-04-24T23:12:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb6800a2b90 0x7fb6800a5050 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698107460 0x7fb69819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb698104b30 0x7fb698197f90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 --2- 172.21.15.102:0/2051320025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb698102200 0x7fb698197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 >> 172.21.15.102:0/2051320025 conn(0x7fb6980fbbf0 msgr2=0x7fb6981040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 shutdown_connections 2024-04-24T23:12:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.387+0000 7fb69e35a640 1 -- 172.21.15.102:0/2051320025 wait complete. 2024-04-24T23:12:24.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:24 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2840779414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:24.400 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:24.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 -- 172.21.15.102:0/1572597489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d641006a0 msgr2=0x7f1d64100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1572597489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d641006a0 0x7f1d64100ac0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009900 tx=0x7f1d5402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:24.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 -- 172.21.15.102:0/1572597489 shutdown_connections 2024-04-24T23:12:24.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1572597489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d64102250 0x7f1d64108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1572597489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d64101890 0x7f1d64101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1572597489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d641006a0 0x7f1d64100ac0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 -- 172.21.15.102:0/1572597489 >> 172.21.15.102:0/1572597489 conn(0x7f1d640fbc10 msgr2=0x7f1d640fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 -- 172.21.15.102:0/1572597489 shutdown_connections 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 -- 172.21.15.102:0/1572597489 wait complete. 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 Processor -- start 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.499+0000 7f1d6c709640 1 -- start start 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6c709640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d641006a0 0x7f1d64197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6c709640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d64101890 0x7f1d641980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6c709640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d64102250 0x7f1d6419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6c709640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410c2a0 con 0x7f1d64101890 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6c709640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410c120 con 0x7f1d64102250 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6c709640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410c420 con 0x7f1d641006a0 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d69ca4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d64101890 0x7f1d641980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d69ca4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d64101890 0x7f1d641980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60796/0 (socket says 172.21.15.102:60796) 2024-04-24T23:12:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d69ca4640 1 -- 172.21.15.102:0/1724197449 learned_addr learned my addr 172.21.15.102:0/1724197449 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6aca6640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d64102250 0x7f1d6419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d69ca4640 1 -- 172.21.15.102:0/1724197449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d641006a0 msgr2=0x7f1d64197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d69ca4640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d641006a0 0x7f1d64197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d69ca4640 1 -- 172.21.15.102:0/1724197449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d64102250 msgr2=0x7f1d6419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d69ca4640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d64102250 0x7f1d6419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d69ca4640 1 -- 172.21.15.102:0/1724197449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6419f7d0 con 0x7f1d64101890 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6aca6640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d64102250 0x7f1d6419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d69ca4640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d64101890 0x7f1d641980f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000b4a0 tx=0x7f1d6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d5b7fe640 1 -- 172.21.15.102:0/1724197449 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60014070 con 0x7f1d64101890 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d5b7fe640 1 -- 172.21.15.102:0/1724197449 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d60010ac0 con 0x7f1d64101890 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6419fac0 con 0x7f1d64101890 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d5b7fe640 1 -- 172.21.15.102:0/1724197449 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60017c30 con 0x7f1d64101890 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.500+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641a0010 con 0x7f1d64101890 2024-04-24T23:12:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.501+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d641a02a0 con 0x7f1d64101890 2024-04-24T23:12:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.503+0000 7f1d5b7fe640 1 -- 172.21.15.102:0/1724197449 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1d60017070 con 0x7f1d64101890 2024-04-24T23:12:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.504+0000 7f1d5b7fe640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d500a2b90 0x7f1d500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.504+0000 7f1d6a4a5640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d500a2b90 0x7f1d500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.504+0000 7f1d5b7fe640 1 -- 172.21.15.102:0/1724197449 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f1d600ce810 con 0x7f1d64101890 2024-04-24T23:12:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.504+0000 7f1d6a4a5640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d500a2b90 0x7f1d500a5050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d5402a930 tx=0x7f1d540314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.507+0000 7f1d5b7fe640 1 -- 172.21.15.102:0/1724197449 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d6009b760 con 0x7f1d64101890 2024-04-24T23:12:24.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:24 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2840779414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:24.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.681+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1d641a0510 con 0x7f1d64101890 2024-04-24T23:12:24.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.682+0000 7f1d5b7fe640 1 -- 172.21.15.102:0/1724197449 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f1d6009b060 con 0x7f1d64101890 2024-04-24T23:12:24.688 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d500a2b90 msgr2=0x7f1d500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d500a2b90 0x7f1d500a5050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d5402a930 tx=0x7f1d540314a0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d64101890 msgr2=0x7f1d641980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d64101890 0x7f1d641980f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000b4a0 tx=0x7f1d6000b960 comp rx=0 tx=0).stop 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 shutdown_connections 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f1d500a2b90 0x7f1d500a5050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d64102250 0x7f1d6419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d64101890 0x7f1d641980f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 --2- 172.21.15.102:0/1724197449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d641006a0 0x7f1d64197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 >> 172.21.15.102:0/1724197449 conn(0x7f1d640fbc10 msgr2=0x7f1d64069db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 shutdown_connections 2024-04-24T23:12:24.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.684+0000 7f1d6c709640 1 -- 172.21.15.102:0/1724197449 wait complete. 2024-04-24T23:12:24.697 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:24.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:24 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2840779414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:24.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 -- 172.21.15.102:0/3351757178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20102220 msgr2=0x7f2f20104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3351757178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20102220 0x7f2f20104610 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009900 tx=0x7f2f1402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:24.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 -- 172.21.15.102:0/3351757178 shutdown_connections 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3351757178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20107480 0x7f2f201098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3351757178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20104b50 0x7f2f20106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3351757178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20102220 0x7f2f20104610 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 -- 172.21.15.102:0/3351757178 >> 172.21.15.102:0/3351757178 conn(0x7f2f200fbc30 msgr2=0x7f2f200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 -- 172.21.15.102:0/3351757178 shutdown_connections 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 -- 172.21.15.102:0/3351757178 wait complete. 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 Processor -- start 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 -- start start 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.796+0000 7f2f256a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20102220 0x7f2f20197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f256a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20104b50 0x7f2f20197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f256a7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20107480 0x7f2f2019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f256a7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010d2f0 con 0x7f2f20107480 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f256a7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010d170 con 0x7f2f20102220 2024-04-24T23:12:24.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f256a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010d470 con 0x7f2f20104b50 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20107480 0x7f2f2019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20107480 0x7f2f2019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60822/0 (socket says 172.21.15.102:60822) 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1f7fe640 1 -- 172.21.15.102:0/3964581481 learned_addr learned my addr 172.21.15.102:0/3964581481 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1effd640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20102220 0x7f2f20197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1e7fc640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20104b50 0x7f2f20197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1f7fe640 1 -- 172.21.15.102:0/3964581481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20104b50 msgr2=0x7f2f20197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1f7fe640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20104b50 0x7f2f20197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1f7fe640 1 -- 172.21.15.102:0/3964581481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20102220 msgr2=0x7f2f20197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1f7fe640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20102220 0x7f2f20197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1f7fe640 1 -- 172.21.15.102:0/3964581481 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2019f680 con 0x7f2f20107480 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f1f7fe640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20107480 0x7f2f2019f010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800eaa0 tx=0x7f2f1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f07fff640 1 -- 172.21.15.102:0/3964581481 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800cdb0 con 0x7f2f20107480 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f07fff640 1 -- 172.21.15.102:0/3964581481 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f18005020 con 0x7f2f20107480 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2019f970 con 0x7f2f20107480 2024-04-24T23:12:24.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f07fff640 1 -- 172.21.15.102:0/3964581481 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f180116d0 con 0x7f2f20107480 2024-04-24T23:12:24.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.797+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2019fec0 con 0x7f2f20107480 2024-04-24T23:12:24.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.798+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f200fd720 con 0x7f2f20107480 2024-04-24T23:12:24.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.799+0000 7f2f07fff640 1 -- 172.21.15.102:0/3964581481 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2f180051d0 con 0x7f2f20107480 2024-04-24T23:12:24.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.800+0000 7f2f07fff640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2f080a2b90 0x7f2f080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.800+0000 7f2f1effd640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2f080a2b90 0x7f2f080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.800+0000 7f2f07fff640 1 -- 172.21.15.102:0/3964581481 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f2f180cf200 con 0x7f2f20107480 2024-04-24T23:12:24.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.800+0000 7f2f1effd640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2f080a2b90 0x7f2f080a5050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2f14006010 tx=0x7f2f14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.804+0000 7f2f07fff640 1 -- 172.21.15.102:0/3964581481 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2f18016030 con 0x7f2f20107480 2024-04-24T23:12:24.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.982+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f20062370 con 0x7f2f20107480 2024-04-24T23:12:24.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.982+0000 7f2f07fff640 1 -- 172.21.15.102:0/3964581481 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f2f1809c150 con 0x7f2f20107480 2024-04-24T23:12:24.984 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-24T23:12:24.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2f080a2b90 msgr2=0x7f2f080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2f080a2b90 0x7f2f080a5050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2f14006010 tx=0x7f2f14000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20107480 msgr2=0x7f2f2019f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20107480 0x7f2f2019f010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800eaa0 tx=0x7f2f1800ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:24.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 shutdown_connections 2024-04-24T23:12:24.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2f080a2b90 0x7f2f080a5050 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f20107480 0x7f2f2019f010 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20104b50 0x7f2f20197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 --2- 172.21.15.102:0/3964581481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20102220 0x7f2f20197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 >> 172.21.15.102:0/3964581481 conn(0x7f2f200fbc30 msgr2=0x7f2f20102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 shutdown_connections 2024-04-24T23:12:24.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:24.984+0000 7f2f256a7640 1 -- 172.21.15.102:0/3964581481 wait complete. 2024-04-24T23:12:24.998 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:25.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.093+0000 7efddd314640 1 -- 172.21.15.102:0/1450459801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8103340 msgr2=0x7efdd8109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.093+0000 7efddd314640 1 --2- 172.21.15.102:0/1450459801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8103340 0x7efdd8109bd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009ca0 tx=0x7efdd002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.093+0000 7efddd314640 1 -- 172.21.15.102:0/1450459801 shutdown_connections 2024-04-24T23:12:25.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.093+0000 7efddd314640 1 --2- 172.21.15.102:0/1450459801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8103340 0x7efdd8109bd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.093+0000 7efddd314640 1 --2- 172.21.15.102:0/1450459801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdd8102980 0x7efdd8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.093+0000 7efddd314640 1 --2- 172.21.15.102:0/1450459801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd8101790 0x7efdd8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.093+0000 7efddd314640 1 -- 172.21.15.102:0/1450459801 >> 172.21.15.102:0/1450459801 conn(0x7efdd80fcd40 msgr2=0x7efdd80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.093+0000 7efddd314640 1 -- 172.21.15.102:0/1450459801 shutdown_connections 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.093+0000 7efddd314640 1 -- 172.21.15.102:0/1450459801 wait complete. 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efddd314640 1 Processor -- start 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efddd314640 1 -- start start 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efddd314640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdd8101790 0x7efdd80754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efddd314640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8102980 0x7efdd8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efddd314640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd8103340 0x7efdd8074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efddd314640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efdd810d2a0 con 0x7efdd8102980 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efddd314640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efdd810d120 con 0x7efdd8103340 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efddd314640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efdd810d420 con 0x7efdd8101790 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efdd67fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8102980 0x7efdd8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efdd67fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8102980 0x7efdd8073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60842/0 (socket says 172.21.15.102:60842) 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efdd67fc640 1 -- 172.21.15.102:0/1523630107 learned_addr learned my addr 172.21.15.102:0/1523630107 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efdd6ffd640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdd8101790 0x7efdd80754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efdd67fc640 1 -- 172.21.15.102:0/1523630107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdd8101790 msgr2=0x7efdd80754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efdd67fc640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdd8101790 0x7efdd80754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efdd67fc640 1 -- 172.21.15.102:0/1523630107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd8103340 msgr2=0x7efdd8074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efdd67fc640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd8103340 0x7efdd8074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.094+0000 7efdd67fc640 1 -- 172.21.15.102:0/1523630107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd8074900 con 0x7efdd8102980 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.095+0000 7efdd67fc640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8102980 0x7efdd8073b30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c920 tx=0x7efdcc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.095+0000 7efdbffff640 1 -- 172.21.15.102:0/1523630107 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc007c70 con 0x7efdd8102980 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.095+0000 7efdbffff640 1 -- 172.21.15.102:0/1523630107 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdcc007e10 con 0x7efdd8102980 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.095+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd81a1a00 con 0x7efdd8102980 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.095+0000 7efdbffff640 1 -- 172.21.15.102:0/1523630107 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc018820 con 0x7efdd8102980 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.095+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd81a1fa0 con 0x7efdd8102980 2024-04-24T23:12:25.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.096+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd8108a50 con 0x7efdd8102980 2024-04-24T23:12:25.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.097+0000 7efdbffff640 1 -- 172.21.15.102:0/1523630107 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efdcc0189c0 con 0x7efdd8102980 2024-04-24T23:12:25.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.098+0000 7efdbffff640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efdc00a2c10 0x7efdc00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.098+0000 7efdd6ffd640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efdc00a2c10 0x7efdc00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.098+0000 7efdbffff640 1 -- 172.21.15.102:0/1523630107 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7efdcc0cf1f0 con 0x7efdd8102980 2024-04-24T23:12:25.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.098+0000 7efdd6ffd640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efdc00a2c10 0x7efdc00a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efdc8005d00 tx=0x7efdc8005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.102+0000 7efdbffff640 1 -- 172.21.15.102:0/1523630107 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efdcc014030 con 0x7efdd8102980 2024-04-24T23:12:25.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.314+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdd802d170 con 0x7efdd8102980 2024-04-24T23:12:25.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.315+0000 7efdbffff640 1 -- 172.21.15.102:0/1523630107 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7efdcc09c140 con 0x7efdd8102980 2024-04-24T23:12:25.317 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:25.317 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:11:00.643113+0000","modified":"2024-04-24T23:11:04.856565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/1258581930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1258581930},{"type":"v1","addr":"172.21.15.161:6839","nonce":1258581930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24473,24497,24472]},"id":1}]} 2024-04-24T23:12:25.317 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efdc00a2c10 msgr2=0x7efdc00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efdc00a2c10 0x7efdc00a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efdc8005d00 tx=0x7efdc8005c50 comp rx=0 tx=0).stop 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8102980 msgr2=0x7efdd8073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8102980 0x7efdd8073b30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c920 tx=0x7efdcc00cde0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 shutdown_connections 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efdc00a2c10 0x7efdc00a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd8103340 0x7efdd8074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdd8102980 0x7efdd8073b30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 --2- 172.21.15.102:0/1523630107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdd8101790 0x7efdd80754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 >> 172.21.15.102:0/1523630107 conn(0x7efdd80fcd40 msgr2=0x7efdd8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 shutdown_connections 2024-04-24T23:12:25.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.317+0000 7efddd314640 1 -- 172.21.15.102:0/1523630107 wait complete. 2024-04-24T23:12:25.330 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:25.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.425+0000 7fee4d800640 1 -- 172.21.15.102:0/3313371256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee48102090 msgr2=0x7fee48108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.425+0000 7fee4d800640 1 --2- 172.21.15.102:0/3313371256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee48102090 0x7fee48108920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fee38009900 tx=0x7fee3802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:25.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.425+0000 7fee4d800640 1 -- 172.21.15.102:0/3313371256 shutdown_connections 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.425+0000 7fee4d800640 1 --2- 172.21.15.102:0/3313371256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee48102090 0x7fee48108920 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.425+0000 7fee4d800640 1 --2- 172.21.15.102:0/3313371256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee481016d0 0x7fee48101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.425+0000 7fee4d800640 1 --2- 172.21.15.102:0/3313371256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee481004e0 0x7fee48100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.425+0000 7fee4d800640 1 -- 172.21.15.102:0/3313371256 >> 172.21.15.102:0/3313371256 conn(0x7fee480fba90 msgr2=0x7fee480fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.425+0000 7fee4d800640 1 -- 172.21.15.102:0/3313371256 shutdown_connections 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.425+0000 7fee4d800640 1 -- 172.21.15.102:0/3313371256 wait complete. 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.426+0000 7fee4d800640 1 Processor -- start 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.426+0000 7fee4d800640 1 -- start start 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.426+0000 7fee4d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee481004e0 0x7fee4806a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.426+0000 7fee4d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee481016d0 0x7fee4819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.426+0000 7fee4d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee48102090 0x7fee4819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.426+0000 7fee4d800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fee4810c020 con 0x7fee481016d0 2024-04-24T23:12:25.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.426+0000 7fee4d800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee4810bea0 con 0x7fee48102090 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.426+0000 7fee4d800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee4810c1a0 con 0x7fee481004e0 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee467fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee481016d0 0x7fee4819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee467fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee481016d0 0x7fee4819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60860/0 (socket says 172.21.15.102:60860) 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee467fc640 1 -- 172.21.15.102:0/2522919532 learned_addr learned my addr 172.21.15.102:0/2522919532 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee477fe640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee48102090 0x7fee4819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee46ffd640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee481004e0 0x7fee4806a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee467fc640 1 -- 172.21.15.102:0/2522919532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee481004e0 msgr2=0x7fee4806a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee467fc640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee481004e0 0x7fee4806a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee467fc640 1 -- 172.21.15.102:0/2522919532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee48102090 msgr2=0x7fee4819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee467fc640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee48102090 0x7fee4819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee467fc640 1 -- 172.21.15.102:0/2522919532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4819f400 con 0x7fee481016d0 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee467fc640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee481016d0 0x7fee4819c4d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00ea80 tx=0x7fee3c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee2ffff640 1 -- 172.21.15.102:0/2522919532 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00ce40 con 0x7fee481016d0 2024-04-24T23:12:25.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee2ffff640 1 -- 172.21.15.102:0/2522919532 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee3c005020 con 0x7fee481016d0 2024-04-24T23:12:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee2ffff640 1 -- 172.21.15.102:0/2522919532 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0116d0 con 0x7fee481016d0 2024-04-24T23:12:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4819f6f0 con 0x7fee481016d0 2024-04-24T23:12:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.427+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4819fbf0 con 0x7fee481016d0 2024-04-24T23:12:25.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.428+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee48107820 con 0x7fee481016d0 2024-04-24T23:12:25.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.429+0000 7fee2ffff640 1 -- 172.21.15.102:0/2522919532 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fee3c0051d0 con 0x7fee481016d0 2024-04-24T23:12:25.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.430+0000 7fee2ffff640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fee300a2b90 0x7fee300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.430+0000 7fee46ffd640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fee300a2b90 0x7fee300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.430+0000 7fee2ffff640 1 -- 172.21.15.102:0/2522919532 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fee3c0cf380 con 0x7fee481016d0 2024-04-24T23:12:25.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.431+0000 7fee46ffd640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fee300a2b90 0x7fee300a5050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee40005fd0 tx=0x7fee40005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.435+0000 7fee2ffff640 1 -- 172.21.15.102:0/2522919532 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fee3c016030 con 0x7fee481016d0 2024-04-24T23:12:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:25 smithi161 ceph-mon[32709]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T23:12:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:25 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2051320025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:25 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1724197449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:25 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3964581481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:25 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1523630107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:25.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.610+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee48062340 con 0x7fee481016d0 2024-04-24T23:12:25.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.612+0000 7fee2ffff640 1 -- 172.21.15.102:0/2522919532 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22720 (secure 0 0 0) 0x7fee3c09c2d0 con 0x7fee481016d0 2024-04-24T23:12:25.614 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:25.614 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:11:49.323161+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fee300a2b90 msgr2=0x7fee300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fee300a2b90 0x7fee300a5050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee40005fd0 tx=0x7fee40005850 comp rx=0 tx=0).stop 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee481016d0 msgr2=0x7fee4819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee481016d0 0x7fee4819c4d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00ea80 tx=0x7fee3c00ef40 comp rx=0 tx=0).stop 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 shutdown_connections 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fee300a2b90 0x7fee300a5050 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee48102090 0x7fee4819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee481016d0 0x7fee4819c4d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 --2- 172.21.15.102:0/2522919532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee481004e0 0x7fee4806a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 >> 172.21.15.102:0/2522919532 conn(0x7fee480fba90 msgr2=0x7fee481049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 shutdown_connections 2024-04-24T23:12:25.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.614+0000 7fee4d800640 1 -- 172.21.15.102:0/2522919532 wait complete. 2024-04-24T23:12:25.626 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:25 smithi102 ceph-mon[29216]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T23:12:25.626 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:25 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2051320025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.626 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:25 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1724197449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.626 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:25 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3964581481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.626 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:25 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1523630107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:25.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:58.730586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:59.855279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:12:25.628 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:25.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 -- 172.21.15.102:0/2838336194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028101870 msgr2=0x7fc028101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 --2- 172.21.15.102:0/2838336194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028101870 0x7fc028101cf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc018009900 tx=0x7fc01802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:25.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 -- 172.21.15.102:0/2838336194 shutdown_connections 2024-04-24T23:12:25.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 --2- 172.21.15.102:0/2838336194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028102230 0x7fc028108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 --2- 172.21.15.102:0/2838336194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028101870 0x7fc028101cf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 --2- 172.21.15.102:0/2838336194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc028100680 0x7fc028100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 -- 172.21.15.102:0/2838336194 >> 172.21.15.102:0/2838336194 conn(0x7fc0280fbc30 msgr2=0x7fc0280fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 -- 172.21.15.102:0/2838336194 shutdown_connections 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 -- 172.21.15.102:0/2838336194 wait complete. 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 Processor -- start 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 -- start start 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028100680 0x7fc028197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028101870 0x7fc028197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc028102230 0x7fc02819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc02810c260 con 0x7fc028100680 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc02810c0e0 con 0x7fc028102230 2024-04-24T23:12:25.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc030161640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc02810c3e0 con 0x7fc028101870 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc02defd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028100680 0x7fc028197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.726+0000 7fc02defd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028100680 0x7fc028197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60872/0 (socket says 172.21.15.102:60872) 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02defd640 1 -- 172.21.15.102:0/211265942 learned_addr learned my addr 172.21.15.102:0/211265942 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02d6fc640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028101870 0x7fc028197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02e6fe640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc028102230 0x7fc02819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02defd640 1 -- 172.21.15.102:0/211265942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028101870 msgr2=0x7fc028197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02defd640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028101870 0x7fc028197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02defd640 1 -- 172.21.15.102:0/211265942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc028102230 msgr2=0x7fc02819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02defd640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc028102230 0x7fc02819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02defd640 1 -- 172.21.15.102:0/211265942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02819f5b0 con 0x7fc028100680 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02e6fe640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc028102230 0x7fc02819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02d6fc640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028101870 0x7fc028197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc02defd640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028100680 0x7fc028197990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc02400c970 tx=0x7fc02400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc01effd640 1 -- 172.21.15.102:0/211265942 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024007c10 con 0x7fc028100680 2024-04-24T23:12:25.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02819f8a0 con 0x7fc028100680 2024-04-24T23:12:25.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc01effd640 1 -- 172.21.15.102:0/211265942 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc024007db0 con 0x7fc028100680 2024-04-24T23:12:25.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc01effd640 1 -- 172.21.15.102:0/211265942 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02400f7d0 con 0x7fc028100680 2024-04-24T23:12:25.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.727+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02819fe40 con 0x7fc028100680 2024-04-24T23:12:25.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.728+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0280689b0 con 0x7fc028100680 2024-04-24T23:12:25.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.729+0000 7fc01effd640 1 -- 172.21.15.102:0/211265942 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc024018480 con 0x7fc028100680 2024-04-24T23:12:25.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.729+0000 7fc01effd640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc0140a2b90 0x7fc0140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.730+0000 7fc02d6fc640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc0140a2b90 0x7fc0140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.730+0000 7fc01effd640 1 -- 172.21.15.102:0/211265942 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc0240cf450 con 0x7fc028100680 2024-04-24T23:12:25.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.730+0000 7fc02d6fc640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc0140a2b90 0x7fc0140a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc0180314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.734+0000 7fc01effd640 1 -- 172.21.15.102:0/211265942 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc02409c430 con 0x7fc028100680 2024-04-24T23:12:25.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:25 smithi183 ceph-mon[33137]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T23:12:25.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:25 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2051320025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:25 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1724197449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:25 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3964581481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:25 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1523630107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:25.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.911+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc0281a0880 con 0x7fc028100680 2024-04-24T23:12:25.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.911+0000 7fc01effd640 1 -- 172.21.15.102:0/211265942 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fc02409bd30 con 0x7fc028100680 2024-04-24T23:12:25.913 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:12:25.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc0140a2b90 msgr2=0x7fc0140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc0140a2b90 0x7fc0140a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc0180314a0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028100680 msgr2=0x7fc028197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028100680 0x7fc028197990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc02400c970 tx=0x7fc02400ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 shutdown_connections 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc0140a2b90 0x7fc0140a5050 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc028102230 0x7fc02819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028101870 0x7fc028197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 --2- 172.21.15.102:0/211265942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc028100680 0x7fc028197990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 >> 172.21.15.102:0/211265942 conn(0x7fc0280fbc30 msgr2=0x7fc0280fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 shutdown_connections 2024-04-24T23:12:25.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:25.913+0000 7fc030161640 1 -- 172.21.15.102:0/211265942 wait complete. 2024-04-24T23:12:25.926 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:26.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.025+0000 7f8250010640 1 -- 172.21.15.102:0/4116523593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82480ffa50 msgr2=0x7f82480ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.025+0000 7f8250010640 1 --2- 172.21.15.102:0/4116523593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82480ffa50 0x7f82480ffe70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8244009930 tx=0x7f824402a420 comp rx=0 tx=0).stop 2024-04-24T23:12:26.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.025+0000 7f8250010640 1 -- 172.21.15.102:0/4116523593 shutdown_connections 2024-04-24T23:12:26.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.025+0000 7f8250010640 1 --2- 172.21.15.102:0/4116523593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82481033e0 0x7f8248109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.025+0000 7f8250010640 1 --2- 172.21.15.102:0/4116523593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8248102a20 0x7f8248102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.025+0000 7f8250010640 1 --2- 172.21.15.102:0/4116523593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82480ffa50 0x7f82480ffe70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.025+0000 7f8250010640 1 -- 172.21.15.102:0/4116523593 >> 172.21.15.102:0/4116523593 conn(0x7f82480fb380 msgr2=0x7f82480fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.025+0000 7f8250010640 1 -- 172.21.15.102:0/4116523593 shutdown_connections 2024-04-24T23:12:26.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.025+0000 7f8250010640 1 -- 172.21.15.102:0/4116523593 wait complete. 2024-04-24T23:12:26.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 Processor -- start 2024-04-24T23:12:26.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 -- start start 2024-04-24T23:12:26.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82480ffa50 0x7f8248102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248102a20 0x7f82481009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82481033e0 0x7f8248100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f824810d300 con 0x7f82481033e0 2024-04-24T23:12:26.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f824810d180 con 0x7f8248102a20 2024-04-24T23:12:26.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f824810d480 con 0x7f82480ffa50 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824e5ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82481033e0 0x7f8248100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824e5ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82481033e0 0x7f8248100f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60904/0 (socket says 172.21.15.102:60904) 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824e5ad640 1 -- 172.21.15.102:0/3699117190 learned_addr learned my addr 172.21.15.102:0/3699117190 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824ddac640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82480ffa50 0x7f8248102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824e5ad640 1 -- 172.21.15.102:0/3699117190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82480ffa50 msgr2=0x7f8248102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824e5ad640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82480ffa50 0x7f8248102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824e5ad640 1 -- 172.21.15.102:0/3699117190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248102a20 msgr2=0x7f82481009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824d5ab640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248102a20 0x7f82481009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824e5ad640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248102a20 0x7f82481009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824e5ad640 1 -- 172.21.15.102:0/3699117190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481017a0 con 0x7f82481033e0 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f824e5ad640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82481033e0 0x7f8248100f10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f824000eaa0 tx=0x7f824000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f823effd640 1 -- 172.21.15.102:0/3699117190 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824000cdb0 con 0x7f82481033e0 2024-04-24T23:12:26.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f823effd640 1 -- 172.21.15.102:0/3699117190 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8240005020 con 0x7f82481033e0 2024-04-24T23:12:26.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481a1ac0 con 0x7f82481033e0 2024-04-24T23:12:26.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f823effd640 1 -- 172.21.15.102:0/3699117190 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82400116d0 con 0x7f82481033e0 2024-04-24T23:12:26.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.026+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481a2010 con 0x7f82481033e0 2024-04-24T23:12:26.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.027+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8248108b70 con 0x7f82481033e0 2024-04-24T23:12:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.028+0000 7f823effd640 1 -- 172.21.15.102:0/3699117190 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f82400051d0 con 0x7f82481033e0 2024-04-24T23:12:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.029+0000 7f823effd640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82340a2c10 0x7f82340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.029+0000 7f824ddac640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82340a2c10 0x7f82340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.029+0000 7f823effd640 1 -- 172.21.15.102:0/3699117190 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f82400cf2d0 con 0x7f82481033e0 2024-04-24T23:12:26.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.029+0000 7f824ddac640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82340a2c10 0x7f82340a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8244009900 tx=0x7f8244005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.034+0000 7f823effd640 1 -- 172.21.15.102:0/3699117190 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8240016030 con 0x7f82481033e0 2024-04-24T23:12:26.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.207+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82480623e0 con 0x7f82481033e0 2024-04-24T23:12:26.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.208+0000 7f823effd640 1 -- 172.21.15.102:0/3699117190 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f824009c220 con 0x7f82481033e0 2024-04-24T23:12:26.210 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-24T23:12:26.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.209+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82340a2c10 msgr2=0x7f82340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.209+0000 7f8250010640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82340a2c10 0x7f82340a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8244009900 tx=0x7f8244005fd0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.209+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82481033e0 msgr2=0x7f8248100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.209+0000 7f8250010640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82481033e0 0x7f8248100f10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f824000eaa0 tx=0x7f824000ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:26.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.209+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 shutdown_connections 2024-04-24T23:12:26.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.209+0000 7f8250010640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82340a2c10 0x7f82340a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.209+0000 7f8250010640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82481033e0 0x7f8248100f10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.209+0000 7f8250010640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248102a20 0x7f82481009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.209+0000 7f8250010640 1 --2- 172.21.15.102:0/3699117190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82480ffa50 0x7f8248102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.210+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 >> 172.21.15.102:0/3699117190 conn(0x7f82480fb380 msgr2=0x7f8248105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.210+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 shutdown_connections 2024-04-24T23:12:26.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.210+0000 7f8250010640 1 -- 172.21.15.102:0/3699117190 wait complete. 2024-04-24T23:12:26.223 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:26.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.323+0000 7fc272c4d640 1 -- 172.21.15.102:0/492565270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c102250 msgr2=0x7fc26c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.323+0000 7fc272c4d640 1 --2- 172.21.15.102:0/492565270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c102250 0x7fc26c108ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc264009ca0 tx=0x7fc26402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.323+0000 7fc272c4d640 1 -- 172.21.15.102:0/492565270 shutdown_connections 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.323+0000 7fc272c4d640 1 --2- 172.21.15.102:0/492565270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c102250 0x7fc26c108ae0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.323+0000 7fc272c4d640 1 --2- 172.21.15.102:0/492565270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc26c101890 0x7fc26c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.323+0000 7fc272c4d640 1 --2- 172.21.15.102:0/492565270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc26c1006a0 0x7fc26c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.323+0000 7fc272c4d640 1 -- 172.21.15.102:0/492565270 >> 172.21.15.102:0/492565270 conn(0x7fc26c0fbc30 msgr2=0x7fc26c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.323+0000 7fc272c4d640 1 -- 172.21.15.102:0/492565270 shutdown_connections 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.323+0000 7fc272c4d640 1 -- 172.21.15.102:0/492565270 wait complete. 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc272c4d640 1 Processor -- start 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc272c4d640 1 -- start start 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc272c4d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc26c1006a0 0x7fc26c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc272c4d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c101890 0x7fc26c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc272c4d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc26c102250 0x7fc26c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc272c4d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c10c220 con 0x7fc26c101890 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc272c4d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c10c0a0 con 0x7fc26c1006a0 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc272c4d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c10c3a0 con 0x7fc26c102250 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc26bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c101890 0x7fc26c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc26bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c101890 0x7fc26c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60928/0 (socket says 172.21.15.102:60928) 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc26bfff640 1 -- 172.21.15.102:0/2210239327 learned_addr learned my addr 172.21.15.102:0/2210239327 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc2709e9640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc26c1006a0 0x7fc26c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc26bfff640 1 -- 172.21.15.102:0/2210239327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc26c102250 msgr2=0x7fc26c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc2711ea640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc26c102250 0x7fc26c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc26bfff640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc26c102250 0x7fc26c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc26bfff640 1 -- 172.21.15.102:0/2210239327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc26c1006a0 msgr2=0x7fc26c197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc26bfff640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc26c1006a0 0x7fc26c197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc26bfff640 1 -- 172.21.15.102:0/2210239327 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c19f5b0 con 0x7fc26c101890 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc2709e9640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc26c1006a0 0x7fc26c197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.324+0000 7fc2711ea640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc26c102250 0x7fc26c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.325+0000 7fc26bfff640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c101890 0x7fc26c197ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c970 tx=0x7fc26000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.325+0000 7fc269ffb640 1 -- 172.21.15.102:0/2210239327 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260007bb0 con 0x7fc26c101890 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.325+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c19f8a0 con 0x7fc26c101890 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.325+0000 7fc269ffb640 1 -- 172.21.15.102:0/2210239327 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc260007d50 con 0x7fc26c101890 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.325+0000 7fc269ffb640 1 -- 172.21.15.102:0/2210239327 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26000f450 con 0x7fc26c101890 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.325+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c19fda0 con 0x7fc26c101890 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.326+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26c1a0060 con 0x7fc26c101890 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.327+0000 7fc269ffb640 1 -- 172.21.15.102:0/2210239327 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc260004f20 con 0x7fc26c101890 2024-04-24T23:12:26.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.327+0000 7fc269ffb640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc2540a2b90 0x7fc2540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.328+0000 7fc2709e9640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc2540a2b90 0x7fc2540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.328+0000 7fc269ffb640 1 -- 172.21.15.102:0/2210239327 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc2600cee80 con 0x7fc26c101890 2024-04-24T23:12:26.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.328+0000 7fc2709e9640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc2540a2b90 0x7fc2540a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc26c1016f0 tx=0x7fc25c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.332+0000 7fc269ffb640 1 -- 172.21.15.102:0/2210239327 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc260014030 con 0x7fc26c101890 2024-04-24T23:12:26.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.507+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc26c1a03f0 con 0x7fc26c101890 2024-04-24T23:12:26.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.508+0000 7fc269ffb640 1 -- 172.21.15.102:0/2210239327 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc26009bdd0 con 0x7fc26c101890 2024-04-24T23:12:26.510 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.509+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc2540a2b90 msgr2=0x7fc2540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.509+0000 7fc272c4d640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc2540a2b90 0x7fc2540a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc26c1016f0 tx=0x7fc25c00a560 comp rx=0 tx=0).stop 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c101890 msgr2=0x7fc26c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c101890 0x7fc26c197ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c970 tx=0x7fc26000ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 shutdown_connections 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fc2540a2b90 0x7fc2540a5050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc26c102250 0x7fc26c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc26c101890 0x7fc26c197ed0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 --2- 172.21.15.102:0/2210239327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc26c1006a0 0x7fc26c197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 >> 172.21.15.102:0/2210239327 conn(0x7fc26c0fbc30 msgr2=0x7fc26c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 shutdown_connections 2024-04-24T23:12:26.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.510+0000 7fc272c4d640 1 -- 172.21.15.102:0/2210239327 wait complete. 2024-04-24T23:12:26.521 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:26 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2522919532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:26.522 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:26 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/211265942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:26.522 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:26 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3699117190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:26.523 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:26.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:26 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2522919532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:26.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:26 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/211265942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:26.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:26 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3699117190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.618+0000 7fb53c8f5640 1 -- 172.21.15.102:0/1066778612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 msgr2=0x7fb538100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.618+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/1066778612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 0x7fb538100ac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb528009900 tx=0x7fb52802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.618+0000 7fb53c8f5640 1 -- 172.21.15.102:0/1066778612 shutdown_connections 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.618+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/1066778612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb538102250 0x7fb538108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.618+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/1066778612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb538101890 0x7fb538101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.618+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/1066778612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 0x7fb538100ac0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.618+0000 7fb53c8f5640 1 -- 172.21.15.102:0/1066778612 >> 172.21.15.102:0/1066778612 conn(0x7fb5380fbc30 msgr2=0x7fb5380fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.618+0000 7fb53c8f5640 1 -- 172.21.15.102:0/1066778612 shutdown_connections 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.618+0000 7fb53c8f5640 1 -- 172.21.15.102:0/1066778612 wait complete. 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53c8f5640 1 Processor -- start 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53c8f5640 1 -- start start 2024-04-24T23:12:26.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53c8f5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 0x7fb53806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53c8f5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb538101890 0x7fb53819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53c8f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb538102250 0x7fb53819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53c8f5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb53810c1a0 con 0x7fb5381006a0 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53c8f5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb53810c020 con 0x7fb538101890 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53c8f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb53810c320 con 0x7fb538102250 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 0x7fb53806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 0x7fb53806a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60656/0 (socket says 172.21.15.102:60656) 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb53659c640 1 -- 172.21.15.102:0/590508876 learned_addr learned my addr 172.21.15.102:0/590508876 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.619+0000 7fb52dd9b640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb538101890 0x7fb53819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb536d9d640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb538102250 0x7fb53819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb53659c640 1 -- 172.21.15.102:0/590508876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb538102250 msgr2=0x7fb53819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb53659c640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb538102250 0x7fb53819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb53659c640 1 -- 172.21.15.102:0/590508876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb538101890 msgr2=0x7fb53819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb53659c640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb538101890 0x7fb53819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb53659c640 1 -- 172.21.15.102:0/590508876 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53819f6a0 con 0x7fb5381006a0 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb53659c640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 0x7fb53806a910 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb5280314e0 tx=0x7fb5280042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb52ffff640 1 -- 172.21.15.102:0/590508876 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52803f030 con 0x7fb5381006a0 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53819f8d0 con 0x7fb5381006a0 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb52ffff640 1 -- 172.21.15.102:0/590508876 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb528003cb0 con 0x7fb5381006a0 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb52ffff640 1 -- 172.21.15.102:0/590508876 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528035070 con 0x7fb5381006a0 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.620+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53819fe10 con 0x7fb5381006a0 2024-04-24T23:12:26.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.621+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb538107ae0 con 0x7fb5381006a0 2024-04-24T23:12:26.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.622+0000 7fb52ffff640 1 -- 172.21.15.102:0/590508876 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb528042030 con 0x7fb5381006a0 2024-04-24T23:12:26.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.623+0000 7fb52ffff640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb5240a2b90 0x7fb5240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.623+0000 7fb52dd9b640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb5240a2b90 0x7fb5240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.623+0000 7fb52ffff640 1 -- 172.21.15.102:0/590508876 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb5280ef720 con 0x7fb5381006a0 2024-04-24T23:12:26.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.623+0000 7fb52dd9b640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb5240a2b90 0x7fb5240a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb520005fd0 tx=0x7fb520005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.627+0000 7fb52ffff640 1 -- 172.21.15.102:0/590508876 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb5280bc670 con 0x7fb5381006a0 2024-04-24T23:12:26.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:26 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2522919532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:26.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:26 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/211265942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:26.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:26 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3699117190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:26.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.841+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb538062340 con 0x7fb5381006a0 2024-04-24T23:12:26.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.841+0000 7fb52ffff640 1 -- 172.21.15.102:0/590508876 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7fb5280bbf70 con 0x7fb5381006a0 2024-04-24T23:12:26.843 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:26.843 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:11:00.643113+0000","modified":"2024-04-24T23:11:04.856565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/1258581930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1258581930},{"type":"v1","addr":"172.21.15.161:6839","nonce":1258581930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24473,24497,24472]},"id":1}]} 2024-04-24T23:12:26.844 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-04-24T23:12:26.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb5240a2b90 msgr2=0x7fb5240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb5240a2b90 0x7fb5240a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb520005fd0 tx=0x7fb520005850 comp rx=0 tx=0).stop 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 msgr2=0x7fb53806a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 0x7fb53806a910 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb5280314e0 tx=0x7fb5280042e0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 shutdown_connections 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb5240a2b90 0x7fb5240a5050 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb538102250 0x7fb53819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb538101890 0x7fb53819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 --2- 172.21.15.102:0/590508876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5381006a0 0x7fb53806a910 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.843+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 >> 172.21.15.102:0/590508876 conn(0x7fb5380fbc30 msgr2=0x7fb538104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.844+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 shutdown_connections 2024-04-24T23:12:26.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.844+0000 7fb53c8f5640 1 -- 172.21.15.102:0/590508876 wait complete. 2024-04-24T23:12:26.857 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:26.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 -- 172.21.15.102:0/2207001028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c101790 msgr2=0x7f462c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 --2- 172.21.15.102:0/2207001028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c101790 0x7f462c101bb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4624009ca0 tx=0x7f462402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 -- 172.21.15.102:0/2207001028 shutdown_connections 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 --2- 172.21.15.102:0/2207001028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c103340 0x7f462c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 --2- 172.21.15.102:0/2207001028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c102980 0x7f462c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 --2- 172.21.15.102:0/2207001028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c101790 0x7f462c101bb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 -- 172.21.15.102:0/2207001028 >> 172.21.15.102:0/2207001028 conn(0x7f462c0fcd20 msgr2=0x7f462c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 -- 172.21.15.102:0/2207001028 shutdown_connections 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 -- 172.21.15.102:0/2207001028 wait complete. 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 Processor -- start 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 -- start start 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c101790 0x7f462c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c102980 0x7f462c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c103340 0x7f462c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.953+0000 7f463364d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f462c10d290 con 0x7f462c102980 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f463364d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f462c10d110 con 0x7f462c103340 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f463364d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f462c10d410 con 0x7f462c101790 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4630be8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c102980 0x7f462c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4630be8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c102980 0x7f462c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60686/0 (socket says 172.21.15.102:60686) 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4630be8640 1 -- 172.21.15.102:0/3248347586 learned_addr learned my addr 172.21.15.102:0/3248347586 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4631bea640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c103340 0x7f462c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f46313e9640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c101790 0x7f462c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4630be8640 1 -- 172.21.15.102:0/3248347586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c101790 msgr2=0x7f462c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4630be8640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c101790 0x7f462c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4630be8640 1 -- 172.21.15.102:0/3248347586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c103340 msgr2=0x7f462c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4630be8640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c103340 0x7f462c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4630be8640 1 -- 172.21.15.102:0/3248347586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c19f620 con 0x7f462c102980 2024-04-24T23:12:26.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4631bea640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c103340 0x7f462c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:26.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f4630be8640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c102980 0x7f462c197f40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d950 tx=0x7f461c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f46227fc640 1 -- 172.21.15.102:0/3248347586 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c017070 con 0x7f462c102980 2024-04-24T23:12:26.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c19f910 con 0x7f462c102980 2024-04-24T23:12:26.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f46227fc640 1 -- 172.21.15.102:0/3248347586 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c009c30 con 0x7f462c102980 2024-04-24T23:12:26.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f46227fc640 1 -- 172.21.15.102:0/3248347586 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c00f620 con 0x7f462c102980 2024-04-24T23:12:26.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.954+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c19fe60 con 0x7f462c102980 2024-04-24T23:12:26.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.955+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c108b20 con 0x7f462c102980 2024-04-24T23:12:26.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.956+0000 7f46227fc640 1 -- 172.21.15.102:0/3248347586 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f461c0109a0 con 0x7f462c102980 2024-04-24T23:12:26.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.957+0000 7f46227fc640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f46180a2b90 0x7f46180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.957+0000 7f46313e9640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f46180a2b90 0x7f46180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.957+0000 7f46227fc640 1 -- 172.21.15.102:0/3248347586 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f461c0cecd0 con 0x7f462c102980 2024-04-24T23:12:26.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.957+0000 7f46313e9640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f46180a2b90 0x7f46180a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4624000f80 tx=0x7f4624004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:26.961+0000 7f46227fc640 1 -- 172.21.15.102:0/3248347586 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f461c009de0 con 0x7f462c102980 2024-04-24T23:12:27.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.137+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f462c02d3c0 con 0x7f462c102980 2024-04-24T23:12:27.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.138+0000 7f46227fc640 1 -- 172.21.15.102:0/3248347586 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22720 (secure 0 0 0) 0x7f461c09bc20 con 0x7f462c102980 2024-04-24T23:12:27.141 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:27.141 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":88,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:11:49.323161+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:27.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.140+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f46180a2b90 msgr2=0x7f46180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.140+0000 7f463364d640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f46180a2b90 0x7f46180a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4624000f80 tx=0x7f4624004a40 comp rx=0 tx=0).stop 2024-04-24T23:12:27.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c102980 msgr2=0x7f462c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c102980 0x7f462c197f40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d950 tx=0x7f461c00de10 comp rx=0 tx=0).stop 2024-04-24T23:12:27.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 shutdown_connections 2024-04-24T23:12:27.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f46180a2b90 0x7f46180a5050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462c103340 0x7f462c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f462c102980 0x7f462c197f40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 --2- 172.21.15.102:0/3248347586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c101790 0x7f462c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 >> 172.21.15.102:0/3248347586 conn(0x7f462c0fcd20 msgr2=0x7f462c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 shutdown_connections 2024-04-24T23:12:27.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.141+0000 7f463364d640 1 -- 172.21.15.102:0/3248347586 wait complete. 2024-04-24T23:12:27.156 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.251+0000 7f5089f14640 1 -- 172.21.15.102:0/2295591102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084100680 msgr2=0x7f5084100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.251+0000 7f5089f14640 1 --2- 172.21.15.102:0/2295591102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084100680 0x7f5084100aa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5074009900 tx=0x7f507402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.251+0000 7f5089f14640 1 -- 172.21.15.102:0/2295591102 shutdown_connections 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.251+0000 7f5089f14640 1 --2- 172.21.15.102:0/2295591102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5084102230 0x7f5084108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.251+0000 7f5089f14640 1 --2- 172.21.15.102:0/2295591102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5084101870 0x7f5084101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.251+0000 7f5089f14640 1 --2- 172.21.15.102:0/2295591102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084100680 0x7f5084100aa0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.251+0000 7f5089f14640 1 -- 172.21.15.102:0/2295591102 >> 172.21.15.102:0/2295591102 conn(0x7f50840fbbf0 msgr2=0x7f50840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.251+0000 7f5089f14640 1 -- 172.21.15.102:0/2295591102 shutdown_connections 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.251+0000 7f5089f14640 1 -- 172.21.15.102:0/2295591102 wait complete. 2024-04-24T23:12:27.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5089f14640 1 Processor -- start 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5089f14640 1 -- start start 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5089f14640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5084100680 0x7f50841979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5089f14640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5084101870 0x7f5084197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5089f14640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084102230 0x7f508419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5089f14640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f508410d310 con 0x7f5084102230 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5089f14640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f508410d190 con 0x7f5084101870 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5089f14640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f508410d490 con 0x7f5084100680 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5083fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084102230 0x7f508419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5083fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084102230 0x7f508419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60714/0 (socket says 172.21.15.102:60714) 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f5083fff640 1 -- 172.21.15.102:0/3599798665 learned_addr learned my addr 172.21.15.102:0/3599798665 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.252+0000 7f50837fe640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5084100680 0x7f50841979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f507affd640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5084101870 0x7f5084197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f5083fff640 1 -- 172.21.15.102:0/3599798665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5084100680 msgr2=0x7f50841979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f5083fff640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5084100680 0x7f50841979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f5083fff640 1 -- 172.21.15.102:0/3599798665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5084101870 msgr2=0x7f5084197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f5083fff640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5084101870 0x7f5084197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f5083fff640 1 -- 172.21.15.102:0/3599798665 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508419f610 con 0x7f5084102230 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f50837fe640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5084100680 0x7f50841979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f5083fff640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084102230 0x7f508419efa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f507c00cce0 tx=0x7f507c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f50817fa640 1 -- 172.21.15.102:0/3599798665 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c00f450 con 0x7f5084102230 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508419f900 con 0x7f5084102230 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f50817fa640 1 -- 172.21.15.102:0/3599798665 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f507c00faa0 con 0x7f5084102230 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f50817fa640 1 -- 172.21.15.102:0/3599798665 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c018b70 con 0x7f5084102230 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.253+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508419fed0 con 0x7f5084102230 2024-04-24T23:12:27.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.254+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5084107a20 con 0x7f5084102230 2024-04-24T23:12:27.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.255+0000 7f50817fa640 1 -- 172.21.15.102:0/3599798665 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f507c018d10 con 0x7f5084102230 2024-04-24T23:12:27.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.255+0000 7f50817fa640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f50700a2b90 0x7f50700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.255+0000 7f50837fe640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f50700a2b90 0x7f50700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.256+0000 7f50817fa640 1 -- 172.21.15.102:0/3599798665 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f507c0cf540 con 0x7f5084102230 2024-04-24T23:12:27.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.256+0000 7f50837fe640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f50700a2b90 0x7f50700a5050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5074000c00 tx=0x7f5074031d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.260+0000 7f50817fa640 1 -- 172.21.15.102:0/3599798665 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f507c014030 con 0x7f5084102230 2024-04-24T23:12:27.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.480+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f508402d170 con 0x7f5084102230 2024-04-24T23:12:27.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.481+0000 7f50817fa640 1 -- 172.21.15.102:0/3599798665 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7f507c09c490 con 0x7f5084102230 2024-04-24T23:12:27.483 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:27.484 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:11:00.643113+0000","modified":"2024-04-24T23:11:04.856565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/1258581930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1258581930},{"type":"v1","addr":"172.21.15.161:6839","nonce":1258581930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24473,24497,24472]},"id":1}]} 2024-04-24T23:12:27.484 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-04-24T23:12:27.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f50700a2b90 msgr2=0x7f50700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f50700a2b90 0x7f50700a5050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5074000c00 tx=0x7f5074031d10 comp rx=0 tx=0).stop 2024-04-24T23:12:27.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084102230 msgr2=0x7f508419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084102230 0x7f508419efa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f507c00cce0 tx=0x7f507c00a620 comp rx=0 tx=0).stop 2024-04-24T23:12:27.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 shutdown_connections 2024-04-24T23:12:27.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f50700a2b90 0x7f50700a5050 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5084102230 0x7f508419efa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5084101870 0x7f5084197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 --2- 172.21.15.102:0/3599798665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5084100680 0x7f50841979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 >> 172.21.15.102:0/3599798665 conn(0x7f50840fbbf0 msgr2=0x7f5084104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 shutdown_connections 2024-04-24T23:12:27.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.483+0000 7f5089f14640 1 -- 172.21.15.102:0/3599798665 wait complete. 2024-04-24T23:12:27.495 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:27 smithi102 ceph-mon[29216]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-24T23:12:27.495 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:27 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2210239327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:27.496 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:27 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/590508876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:27.496 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:27 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3248347586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:27.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T23:11:00.643113+0000', 'modified': '2024-04-24T23:11:04.856565+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24473, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/2266462643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 2266462643}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 2266462643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/2778927873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 2778927873}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 2778927873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/3933349751', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3933349751}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3933349751}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.183:6839/395274952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 395274952}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 395274952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.161:6837/2386140374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 2386140374}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 2386140374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/1258581930', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 1258581930}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 1258581930}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24473, 24497, 24472]} 2024-04-24T23:12:27.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:12:27.498 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:12:27.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 -- 172.21.15.102:0/1530005728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 msgr2=0x7f0e90102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/1530005728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 0x7f0e90102e00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009900 tx=0x7f0e7802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:27.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 -- 172.21.15.102:0/1530005728 shutdown_connections 2024-04-24T23:12:27.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/1530005728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e90103340 0x7f0e90109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/1530005728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 0x7f0e90102e00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/1530005728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90101790 0x7f0e90101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 -- 172.21.15.102:0/1530005728 >> 172.21.15.102:0/1530005728 conn(0x7f0e900fcd20 msgr2=0x7f0e900ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 -- 172.21.15.102:0/1530005728 shutdown_connections 2024-04-24T23:12:27.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 -- 172.21.15.102:0/1530005728 wait complete. 2024-04-24T23:12:27.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.595+0000 7f0e95a5d640 1 Processor -- start 2024-04-24T23:12:27.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e95a5d640 1 -- start start 2024-04-24T23:12:27.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e95a5d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e90101790 0x7f0e901979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e95a5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 0x7f0e90197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e95a5d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90103340 0x7f0e9019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e95a5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9010d2c0 con 0x7f0e90102980 2024-04-24T23:12:27.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e95a5d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9010d140 con 0x7f0e90103340 2024-04-24T23:12:27.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e95a5d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9010d440 con 0x7f0e90101790 2024-04-24T23:12:27.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e87fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 0x7f0e90197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e87fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 0x7f0e90197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60732/0 (socket says 172.21.15.102:60732) 2024-04-24T23:12:27.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e87fff640 1 -- 172.21.15.102:0/3863044791 learned_addr learned my addr 172.21.15.102:0/3863044791 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90103340 0x7f0e9019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e87fff640 1 -- 172.21.15.102:0/3863044791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e90101790 msgr2=0x7f0e901979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e87fff640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e90101790 0x7f0e901979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e87fff640 1 -- 172.21.15.102:0/3863044791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90103340 msgr2=0x7f0e9019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e87fff640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90103340 0x7f0e9019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e87fff640 1 -- 172.21.15.102:0/3863044791 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9019f5f0 con 0x7f0e90102980 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e87fff640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 0x7f0e90197f10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0e78005940 tx=0x7f0e780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e94a5b640 1 -- 172.21.15.102:0/3863044791 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7803f030 con 0x7f0e90102980 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e94a5b640 1 -- 172.21.15.102:0/3863044791 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e78003cb0 con 0x7f0e90102980 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e95a5d640 1 -- 172.21.15.102:0/3863044791 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9019f880 con 0x7f0e90102980 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e94a5b640 1 -- 172.21.15.102:0/3863044791 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78035070 con 0x7f0e90102980 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.596+0000 7f0e95a5d640 1 -- 172.21.15.102:0/3863044791 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9019fdc0 con 0x7f0e90102980 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.598+0000 7f0e94a5b640 1 -- 172.21.15.102:0/3863044791 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0e78042030 con 0x7f0e90102980 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e9876b740 1 Processor -- start 2024-04-24T23:12:27.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e9876b740 1 -- start start 2024-04-24T23:12:27.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e94a5b640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0e7c0a2c10 0x7f0e7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3a4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3a9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3ac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e9876b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc307f00 con 0x55b6dc3a4e30 2024-04-24T23:12:27.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e9876b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc2b0750 con 0x55b6dc3a4440 2024-04-24T23:12:27.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e9876b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc303ec0 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0e7c0a2c10 0x7f0e7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e87fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e87fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3a9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60734/0 (socket says 172.21.15.102:60734) 2024-04-24T23:12:27.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e87fff640 1 -- 172.21.15.102:0/4091419628 learned_addr learned my addr 172.21.15.102:0/4091419628 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e94a5b640 1 -- 172.21.15.102:0/3863044791 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0e780ef690 con 0x7f0e90102980 2024-04-24T23:12:27.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e8effd640 1 --2- 172.21.15.102:0/4091419628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/4091419628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e87fff640 1 -- 172.21.15.102:0/4091419628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a9fa0 msgr2=0x55b6dc3ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e87fff640 1 --2- 172.21.15.102:0/4091419628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3ac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e87fff640 1 -- 172.21.15.102:0/4091419628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 msgr2=0x55b6dc3a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e87fff640 1 --2- 172.21.15.102:0/4091419628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e87fff640 1 -- 172.21.15.102:0/4091419628 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6dc01c2d0 con 0x55b6dc3a4e30 2024-04-24T23:12:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.599+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0e7c0a2c10 0x7f0e7c0a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005fd0 tx=0x7f0e880137d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e87fff640 1 --2- 172.21.15.102:0/4091419628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3a9930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0e780bbe70 tx=0x7f0e78049bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4777d7727be52e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e85ffb640 1 -- 172.21.15.102:0/4091419628 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78049070 con 0x55b6dc3a4e30 2024-04-24T23:12:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e85ffb640 1 -- 172.21.15.102:0/4091419628 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e78041d80 con 0x55b6dc3a4e30 2024-04-24T23:12:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e85ffb640 1 -- 172.21.15.102:0/4091419628 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780876d0 con 0x55b6dc3a4e30 2024-04-24T23:12:27.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e9876b740 1 -- 172.21.15.102:0/4091419628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a4e30 msgr2=0x55b6dc3a9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e9876b740 1 --2- 172.21.15.102:0/4091419628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3a9930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0e780bbe70 tx=0x7f0e78049bd0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e9876b740 1 -- 172.21.15.102:0/4091419628 shutdown_connections 2024-04-24T23:12:27.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e9876b740 1 --2- 172.21.15.102:0/4091419628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e9876b740 1 --2- 172.21.15.102:0/4091419628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3a9930 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e9876b740 1 --2- 172.21.15.102:0/4091419628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e9876b740 1 -- 172.21.15.102:0/4091419628 >> 172.21.15.102:0/4091419628 conn(0x55b6dc39fab0 msgr2=0x55b6dc39feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e9876b740 1 -- 172.21.15.102:0/4091419628 shutdown_connections 2024-04-24T23:12:27.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.600+0000 7f0e9876b740 1 -- 172.21.15.102:0/4091419628 wait complete. 2024-04-24T23:12:27.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e9876b740 1 -- start start 2024-04-24T23:12:27.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e9876b740 1 Processor -- start 2024-04-24T23:12:27.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4440 0x55b6dc3ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e9876b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc307f00 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e9876b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc2c3cf0 con 0x55b6dc3a4e30 2024-04-24T23:12:27.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e9876b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc3acd10 con 0x55b6dc3a4440 2024-04-24T23:12:27.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e9876b740 10 client.0 _ll_register_callbacks cb 0x55b6dc2c36e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:27.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e8f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e8f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60748/0 (socket says 172.21.15.102:60748) 2024-04-24T23:12:27.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/392472230 learned_addr learned my addr 172.21.15.102:0/392472230 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e87fff640 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e8effd640 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4440 0x55b6dc3ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/392472230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4440 msgr2=0x55b6dc3ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4440 0x55b6dc3ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.601+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/392472230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4e30 msgr2=0x55b6dc3aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/392472230 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6dc0e66b0 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e87fff640 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:27.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5a50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0e88011890 tx=0x7f0e88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa53ed194ebbbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e6effd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:27.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e6effd640 1 -- 172.21.15.102:0/392472230 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88024030 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e6effd640 1 -- 172.21.15.102:0/392472230 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0e8800eae0 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e9876b740 1 -- 172.21.15.102:0/392472230 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6dc315440 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e6effd640 1 -- 172.21.15.102:0/392472230 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0e8800ece0 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e9876b740 10 client.15060 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:27.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e9876b740 1 -- 172.21.15.102:0/392472230 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b6dc3b8aa0 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e6effd640 1 -- 172.21.15.102:0/392472230 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b6dc315440 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e6effd640 1 -- 172.21.15.102:0/392472230 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f0e8801d050 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e9876b740 10 client.15060 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:27.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e9876b740 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:12:27.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e9876b740 20 client.15060 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:27.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e9876b740 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x55b6dc3bb260 0x55b6dc3bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e9876b740 4 client.15060 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:27.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.602+0000 7f0e9876b740 1 -- 172.21.15.102:0/392472230 --> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6dc30f790 con 0x55b6dc3bb260 2024-04-24T23:12:27.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.603+0000 7f0e8effd640 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x55b6dc3bb260 0x55b6dc3bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.603+0000 7f0e6effd640 10 client.15060 ms_handle_connect on v2:172.21.15.183:6838/395274952 2024-04-24T23:12:27.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.603+0000 7f0e8effd640 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x55b6dc3bb260 0x55b6dc3bc6d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=986792650ddc6811 server_cookie=9fe17c3f19c4d866 in_seq=0 out_seq=0 2024-04-24T23:12:27.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e6effd640 1 -- 172.21.15.102:0/392472230 <== mds.2 v2:172.21.15.183:6838/395274952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6dc30f790 con 0x55b6dc3bb260 2024-04-24T23:12:27.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e6effd640 10 client.15060 handle_command_reply: tid=0 2024-04-24T23:12:27.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 client.15060 shutdown 2024-04-24T23:12:27.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 -- 172.21.15.102:0/392472230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 msgr2=0x55b6dc3b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5a50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0e88011890 tx=0x7f0e88004ab0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 -- 172.21.15.102:0/392472230 shutdown_connections 2024-04-24T23:12:27.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x55b6dc3bb260 0x55b6dc3bc6d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5a50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e6effd640 0 client.15060 ms_handle_reset on v2:172.21.15.183:6838/395274952 2024-04-24T23:12:27.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 --2- 172.21.15.102:0/392472230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4440 0x55b6dc3ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 -- 172.21.15.102:0/392472230 >> 172.21.15.102:0/392472230 conn(0x55b6dc39fab0 msgr2=0x55b6dc39eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 -- 172.21.15.102:0/392472230 shutdown_connections 2024-04-24T23:12:27.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 1 -- 172.21.15.102:0/392472230 wait complete. 2024-04-24T23:12:27.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.604+0000 7f0e9876b740 20 client.15060 trim_cache size 0 max 16384 2024-04-24T23:12:27.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:27 smithi161 ceph-mon[32709]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-24T23:12:27.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:27 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2210239327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:27.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:27 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/590508876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:27.634 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:27 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3248347586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:27.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e9876b740 1 Processor -- start 2024-04-24T23:12:27.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e9876b740 1 -- start start 2024-04-24T23:12:27.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e9876b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc3baa10 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e9876b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc3bb0e0 con 0x55b6dc3a4440 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e9876b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc3bb7b0 con 0x55b6dc3a4e30 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e8f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e8f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60758/0 (socket says 172.21.15.102:60758) 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/2246585432 learned_addr learned my addr 172.21.15.102:0/2246585432 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/2246585432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4e30 msgr2=0x55b6dc3a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/2246585432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/2246585432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 msgr2=0x55b6dc3a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/2246585432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/2246585432 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6dc3a4270 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.622+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/2246585432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0e88015010 tx=0x7f0e88013e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b936e361203b69dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e857fa640 1 -- 172.21.15.102:0/2246585432 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88024070 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e857fa640 1 -- 172.21.15.102:0/2246585432 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e8800e720 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e857fa640 1 -- 172.21.15.102:0/2246585432 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8801fe20 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e9876b740 1 -- 172.21.15.102:0/2246585432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 msgr2=0x55b6dc3b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e9876b740 1 --2- 172.21.15.102:0/2246585432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0e88015010 tx=0x7f0e88013e20 comp rx=0 tx=0).stop 2024-04-24T23:12:27.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e9876b740 1 -- 172.21.15.102:0/2246585432 shutdown_connections 2024-04-24T23:12:27.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e9876b740 1 --2- 172.21.15.102:0/2246585432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b5970 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e9876b740 1 --2- 172.21.15.102:0/2246585432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e9876b740 1 --2- 172.21.15.102:0/2246585432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e9876b740 1 -- 172.21.15.102:0/2246585432 >> 172.21.15.102:0/2246585432 conn(0x55b6dc39fab0 msgr2=0x55b6dc39fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e9876b740 1 -- 172.21.15.102:0/2246585432 shutdown_connections 2024-04-24T23:12:27.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.623+0000 7f0e9876b740 1 -- 172.21.15.102:0/2246585432 wait complete. 2024-04-24T23:12:27.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e9876b740 1 -- start start 2024-04-24T23:12:27.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e9876b740 1 Processor -- start 2024-04-24T23:12:27.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e9876b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e9876b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc3bc550 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e9876b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc3bbe80 con 0x55b6dc3a4440 2024-04-24T23:12:27.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e9876b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6dc3bcc20 con 0x55b6dc3a4e30 2024-04-24T23:12:27.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e9876b740 10 client.0 _ll_register_callbacks cb 0x7f0e88011590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:27.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e8f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e8f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60774/0 (socket says 172.21.15.102:60774) 2024-04-24T23:12:27.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/1971845142 learned_addr learned my addr 172.21.15.102:0/1971845142 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e87fff640 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e8effd640 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/1971845142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4e30 msgr2=0x55b6dc3b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/1971845142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 msgr2=0x55b6dc3b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.624+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e8f7fe640 1 -- 172.21.15.102:0/1971845142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6dc3b4e00 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e8f7fe640 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b4810 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0e88013ef0 tx=0x7f0e8801fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c775032988748394 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e6effd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:27.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e6effd640 1 -- 172.21.15.102:0/1971845142 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88024040 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e9876b740 1 -- 172.21.15.102:0/1971845142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e8800e720 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e6effd640 1 -- 172.21.15.102:0/1971845142 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0e88020480 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e6effd640 1 -- 172.21.15.102:0/1971845142 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0e88020680 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e9876b740 10 client.15066 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:27.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e9876b740 1 -- 172.21.15.102:0/1971845142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b6dc4212d0 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e6effd640 1 -- 172.21.15.102:0/1971845142 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e8800e720 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e6effd640 1 -- 172.21.15.102:0/1971845142 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f0e8801d070 con 0x55b6dc3a9fa0 2024-04-24T23:12:27.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e9876b740 10 client.15066 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:27.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e9876b740 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:12:27.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e9876b740 20 client.15066 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:27.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e9876b740 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x55b6dc425af0 0x55b6dc424800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e9876b740 4 client.15066 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:27.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.625+0000 7f0e9876b740 1 -- 172.21.15.102:0/1971845142 --> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] -- command(tid 0: {"prefix": "status"}) -- 0x55b6dc28db80 con 0x55b6dc425af0 2024-04-24T23:12:27.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e8effd640 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x55b6dc425af0 0x55b6dc424800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e6effd640 10 client.15066 ms_handle_connect on v2:172.21.15.183:6838/395274952 2024-04-24T23:12:27.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e8effd640 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x55b6dc425af0 0x55b6dc424800 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d205303ddd3313d7 server_cookie=8a55e28defc4807d in_seq=0 out_seq=0 2024-04-24T23:12:27.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e6effd640 1 -- 172.21.15.102:0/1971845142 <== mds.2 v2:172.21.15.183:6838/395274952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b6dc28db80 con 0x55b6dc425af0 2024-04-24T23:12:27.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e6effd640 10 client.15066 handle_command_reply: tid=0 2024-04-24T23:12:27.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 client.15066 shutdown 2024-04-24T23:12:27.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 -- 172.21.15.102:0/1971845142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 msgr2=0x55b6dc3b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b4810 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0e88013ef0 tx=0x7f0e8801fdb0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 -- 172.21.15.102:0/1971845142 shutdown_connections 2024-04-24T23:12:27.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x55b6dc425af0 0x55b6dc424800 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6dc3a9fa0 0x55b6dc3b4810 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e6effd640 0 client.15066 ms_handle_reset on v2:172.21.15.183:6838/395274952 2024-04-24T23:12:27.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6dc3a4e30 0x55b6dc3b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 --2- 172.21.15.102:0/1971845142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b6dc3a4440 0x55b6dc3b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 -- 172.21.15.102:0/1971845142 >> 172.21.15.102:0/1971845142 conn(0x55b6dc39fab0 msgr2=0x55b6dc3ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 -- 172.21.15.102:0/1971845142 shutdown_connections 2024-04-24T23:12:27.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.626+0000 7f0e9876b740 1 -- 172.21.15.102:0/1971845142 wait complete. 2024-04-24T23:12:27.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e9876b740 20 client.15066 trim_cache size 0 max 16384 2024-04-24T23:12:27.652 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:12:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 2, 2024-04-24T23:12:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 24472, 2024-04-24T23:12:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-04-24T23:12:27.652 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-04-24T23:12:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-04-24T23:12:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 83.293218068000002, 2024-04-24T23:12:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 20, 2024-04-24T23:12:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 88, 2024-04-24T23:12:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T23:12:27.653 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 100.851003906 2024-04-24T23:12:27.653 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:12:27.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 -- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0e7c0a2c10 msgr2=0x7f0e7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0e7c0a2c10 0x7f0e7c0a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005fd0 tx=0x7f0e880137d0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 -- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 msgr2=0x7f0e90197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 0x7f0e90197f10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0e78005940 tx=0x7f0e780042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 -- 172.21.15.102:0/3863044791 shutdown_connections 2024-04-24T23:12:27.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0e7c0a2c10 0x7f0e7c0a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90103340 0x7f0e9019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e90102980 0x7f0e90197f10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 --2- 172.21.15.102:0/3863044791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e90101790 0x7f0e901979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 -- 172.21.15.102:0/3863044791 >> 172.21.15.102:0/3863044791 conn(0x7f0e900fcd20 msgr2=0x7f0e90105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 -- 172.21.15.102:0/3863044791 shutdown_connections 2024-04-24T23:12:27.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.627+0000 7f0e95a5d640 1 -- 172.21.15.102:0/3863044791 wait complete. 2024-04-24T23:12:27.658 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-24T23:12:27.664 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:27 smithi183 ceph-mon[33137]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-24T23:12:27.664 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:27 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2210239327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:27.664 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:27 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/590508876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:27.664 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:27 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3248347586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:27.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.754+0000 7f206186d640 1 -- 172.21.15.102:0/815242524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c102240 msgr2=0x7f205c108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.754+0000 7f206186d640 1 --2- 172.21.15.102:0/815242524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c102240 0x7f205c108ad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2054009ca0 tx=0x7f205402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.754+0000 7f206186d640 1 -- 172.21.15.102:0/815242524 shutdown_connections 2024-04-24T23:12:27.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.754+0000 7f206186d640 1 --2- 172.21.15.102:0/815242524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c102240 0x7f205c108ad0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.754+0000 7f206186d640 1 --2- 172.21.15.102:0/815242524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c101880 0x7f205c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.754+0000 7f206186d640 1 --2- 172.21.15.102:0/815242524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f205c100690 0x7f205c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.754+0000 7f206186d640 1 -- 172.21.15.102:0/815242524 >> 172.21.15.102:0/815242524 conn(0x7f205c0fbc00 msgr2=0x7f205c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.754+0000 7f206186d640 1 -- 172.21.15.102:0/815242524 shutdown_connections 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.754+0000 7f206186d640 1 -- 172.21.15.102:0/815242524 wait complete. 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f206186d640 1 Processor -- start 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f206186d640 1 -- start start 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f206186d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f205c100690 0x7f205c199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f206186d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c101880 0x7f205c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f206186d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c102240 0x7f205c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f206186d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10c1c0 con 0x7f205c101880 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f206186d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10c040 con 0x7f205c102240 2024-04-24T23:12:27.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f206186d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10c340 con 0x7f205c100690 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f205a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c101880 0x7f205c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f205a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c101880 0x7f205c19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60790/0 (socket says 172.21.15.102:60790) 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.755+0000 7f205a7fc640 1 -- 172.21.15.102:0/3083008278 learned_addr learned my addr 172.21.15.102:0/3083008278 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f205affd640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f205c100690 0x7f205c199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f205b7fe640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c102240 0x7f205c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f205a7fc640 1 -- 172.21.15.102:0/3083008278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f205c100690 msgr2=0x7f205c199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f205a7fc640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f205c100690 0x7f205c199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f205a7fc640 1 -- 172.21.15.102:0/3083008278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c102240 msgr2=0x7f205c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f205a7fc640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c102240 0x7f205c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f205a7fc640 1 -- 172.21.15.102:0/3083008278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c1a1810 con 0x7f205c101880 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f205a7fc640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c101880 0x7f205c19a130 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f204c00d950 tx=0x7f204c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f206086b640 1 -- 172.21.15.102:0/3083008278 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c017070 con 0x7f205c101880 2024-04-24T23:12:27.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f206086b640 1 -- 172.21.15.102:0/3083008278 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f204c009c30 con 0x7f205c101880 2024-04-24T23:12:27.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f206186d640 1 -- 172.21.15.102:0/3083008278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c1a1b00 con 0x7f205c101880 2024-04-24T23:12:27.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f206086b640 1 -- 172.21.15.102:0/3083008278 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c00f620 con 0x7f205c101880 2024-04-24T23:12:27.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.756+0000 7f206186d640 1 -- 172.21.15.102:0/3083008278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c1a2050 con 0x7f205c101880 2024-04-24T23:12:27.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.758+0000 7f206086b640 1 -- 172.21.15.102:0/3083008278 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f204c0109a0 con 0x7f205c101880 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.758+0000 7f2064577740 1 Processor -- start 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.758+0000 7f2064577740 1 -- start start 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.758+0000 7f206086b640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f20440a2b90 0x7f20440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f2064577740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef259860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f2064577740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef25e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f2064577740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 0x5639ef261430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f2064577740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639ef1bcf00 con 0x5639ef259e30 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f2064577740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5639ef165750 con 0x5639ef259440 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f2064577740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639ef1b8ec0 con 0x5639ef25efa0 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205b7fe640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f20440a2b90 0x7f20440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f206086b640 1 -- 172.21.15.102:0/3083008278 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f204c0cec80 con 0x7f205c101880 2024-04-24T23:12:27.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef25e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef25e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60792/0 (socket says 172.21.15.102:60792) 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205a7fc640 1 -- 172.21.15.102:0/1793961965 learned_addr learned my addr 172.21.15.102:0/1793961965 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205affd640 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef259860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205b7fe640 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 0x5639ef261430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205a7fc640 1 -- 172.21.15.102:0/1793961965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 msgr2=0x5639ef261430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205a7fc640 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 0x5639ef261430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205a7fc640 1 -- 172.21.15.102:0/1793961965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 msgr2=0x5639ef259860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205a7fc640 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef259860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205a7fc640 1 -- 172.21.15.102:0/1793961965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639eeed12d0 con 0x5639ef259e30 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205affd640 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef259860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205b7fe640 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 0x5639ef261430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:27.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205b7fe640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f20440a2b90 0x7f20440a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2054031aa0 tx=0x7f2054031530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f205a7fc640 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef25e930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f204c0cc770 tx=0x7f204c09b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c586f73529aad64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f2041ffb640 1 -- 172.21.15.102:0/1793961965 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c020070 con 0x5639ef259e30 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f2041ffb640 1 -- 172.21.15.102:0/1793961965 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f204c094730 con 0x5639ef259e30 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.759+0000 7f2041ffb640 1 -- 172.21.15.102:0/1793961965 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c0997d0 con 0x5639ef259e30 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 -- 172.21.15.102:0/1793961965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 msgr2=0x5639ef25e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef25e930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f204c0cc770 tx=0x7f204c09b110 comp rx=0 tx=0).stop 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 -- 172.21.15.102:0/1793961965 shutdown_connections 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 0x5639ef261430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef25e930 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 --2- 172.21.15.102:0/1793961965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef259860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 -- 172.21.15.102:0/1793961965 >> 172.21.15.102:0/1793961965 conn(0x5639ef254ab0 msgr2=0x5639ef254eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 -- 172.21.15.102:0/1793961965 shutdown_connections 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 -- 172.21.15.102:0/1793961965 wait complete. 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.760+0000 7f2064577740 1 -- start start 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f2064577740 1 Processor -- start 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f2064577740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef263880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f2064577740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef263dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f2064577740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 0x5639ef26ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f2064577740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639ef261f50 con 0x5639ef259e30 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f2064577740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5639ef178cf0 con 0x5639ef259440 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f2064577740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639ef2620d0 con 0x5639ef25efa0 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f2064577740 10 client.0 _ll_register_callbacks cb 0x5639ef1786e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef263dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef263dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60802/0 (socket says 172.21.15.102:60802) 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205a7fc640 1 -- 172.21.15.102:0/3895223094 learned_addr learned my addr 172.21.15.102:0/3895223094 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205affd640 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef263880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205a7fc640 1 -- 172.21.15.102:0/3895223094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 msgr2=0x5639ef26ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205a7fc640 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 0x5639ef26ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205a7fc640 1 -- 172.21.15.102:0/3895223094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 msgr2=0x5639ef263880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205a7fc640 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef263880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205a7fc640 1 -- 172.21.15.102:0/3895223094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639eef9b6b0 con 0x5639ef259e30 2024-04-24T23:12:27.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f205a7fc640 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef263dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f204c0ccae0 tx=0x7f204c09b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c1706dfc752ded server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f203affd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:27.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f203affd640 1 -- 172.21.15.102:0/3895223094 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c020070 con 0x5639ef259e30 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f203affd640 1 -- 172.21.15.102:0/3895223094 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f204c097b40 con 0x5639ef259e30 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.761+0000 7f2064577740 1 -- 172.21.15.102:0/3895223094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f204c094730 con 0x5639ef259e30 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f203affd640 1 -- 172.21.15.102:0/3895223094 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f204c09a920 con 0x5639ef259e30 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f2064577740 10 client.15078 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f2064577740 1 -- 172.21.15.102:0/3895223094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5639ef26dde0 con 0x5639ef259e30 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f203affd640 1 -- 172.21.15.102:0/3895223094 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f204c094730 con 0x5639ef259e30 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f203affd640 1 -- 172.21.15.102:0/3895223094 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f204c0104d0 con 0x5639ef259e30 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f2064577740 10 client.15078 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f2064577740 10 client.15078 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f2064577740 20 client.15078 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f2064577740 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x5639ef26ff50 0x5639ef2713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f2064577740 4 client.15078 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:27.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f2064577740 1 -- 172.21.15.102:0/3895223094 --> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639ef1ca440 con 0x5639ef26ff50 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.762+0000 7f205affd640 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x5639ef26ff50 0x5639ef2713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.763+0000 7f203affd640 10 client.15078 ms_handle_connect on v2:172.21.15.161:6836/2386140374 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.763+0000 7f205affd640 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x5639ef26ff50 0x5639ef2713c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31a7bb5c2f5cf2d3 server_cookie=2cbb5b244dccb73c in_seq=0 out_seq=0 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.763+0000 7f203affd640 1 -- 172.21.15.102:0/3895223094 <== mds.1 v2:172.21.15.161:6836/2386140374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639ef1ca440 con 0x5639ef26ff50 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.763+0000 7f203affd640 10 client.15078 handle_command_reply: tid=0 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.763+0000 7f2064577740 1 client.15078 shutdown 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.763+0000 7f2064577740 1 -- 172.21.15.102:0/3895223094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 msgr2=0x5639ef263dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.763+0000 7f2064577740 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef263dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f204c0ccae0 tx=0x7f204c09b5c0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f2064577740 1 -- 172.21.15.102:0/3895223094 shutdown_connections 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f2064577740 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x5639ef26ff50 0x5639ef2713c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f2064577740 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef25efa0 0x5639ef26ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f2064577740 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef259e30 0x5639ef263dc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f2064577740 1 --2- 172.21.15.102:0/3895223094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef263880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f203affd640 0 client.15078 ms_handle_reset on v2:172.21.15.161:6836/2386140374 2024-04-24T23:12:27.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f2064577740 1 -- 172.21.15.102:0/3895223094 >> 172.21.15.102:0/3895223094 conn(0x5639ef254ab0 msgr2=0x5639ef253ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f2064577740 1 -- 172.21.15.102:0/3895223094 shutdown_connections 2024-04-24T23:12:27.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f2064577740 1 -- 172.21.15.102:0/3895223094 wait complete. 2024-04-24T23:12:27.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.764+0000 7f2064577740 20 client.15078 trim_cache size 0 max 16384 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.781+0000 7f2064577740 1 Processor -- start 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.781+0000 7f2064577740 1 -- start start 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.781+0000 7f2064577740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef261390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.781+0000 7f2064577740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 0x5639ef26adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.781+0000 7f2064577740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef261da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.781+0000 7f2064577740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639ef26f700 con 0x5639ef25efa0 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.781+0000 7f2064577740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5639ef26fdd0 con 0x5639ef259440 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.781+0000 7f2064577740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639ef2704a0 con 0x5639ef259e30 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.781+0000 7f205b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef261da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef261da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60818/0 (socket says 172.21.15.102:60818) 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205b7fe640 1 -- 172.21.15.102:0/2020745417 learned_addr learned my addr 172.21.15.102:0/2020745417 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205affd640 1 --2- 172.21.15.102:0/2020745417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef261390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205b7fe640 1 -- 172.21.15.102:0/2020745417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 msgr2=0x5639ef26adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205b7fe640 1 --2- 172.21.15.102:0/2020745417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 0x5639ef26adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205b7fe640 1 -- 172.21.15.102:0/2020745417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 msgr2=0x5639ef261390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205b7fe640 1 --2- 172.21.15.102:0/2020745417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef261390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205b7fe640 1 -- 172.21.15.102:0/2020745417 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639ef25e510 con 0x5639ef25efa0 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205affd640 1 --2- 172.21.15.102:0/2020745417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef261390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f205b7fe640 1 --2- 172.21.15.102:0/2020745417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef261da0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2054031db0 tx=0x7f2054045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec649cdb515d6e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f20417fa640 1 -- 172.21.15.102:0/2020745417 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205404e070 con 0x5639ef25efa0 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f20417fa640 1 -- 172.21.15.102:0/2020745417 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20540044e0 con 0x5639ef25efa0 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.782+0000 7f20417fa640 1 -- 172.21.15.102:0/2020745417 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054030b90 con 0x5639ef25efa0 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 -- 172.21.15.102:0/2020745417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 msgr2=0x5639ef261da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 --2- 172.21.15.102:0/2020745417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef261da0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2054031db0 tx=0x7f2054045000 comp rx=0 tx=0).stop 2024-04-24T23:12:27.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 -- 172.21.15.102:0/2020745417 shutdown_connections 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 --2- 172.21.15.102:0/2020745417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef261da0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 --2- 172.21.15.102:0/2020745417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 0x5639ef26adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 --2- 172.21.15.102:0/2020745417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef261390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 -- 172.21.15.102:0/2020745417 >> 172.21.15.102:0/2020745417 conn(0x5639ef254ab0 msgr2=0x5639ef254e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 -- 172.21.15.102:0/2020745417 shutdown_connections 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 -- 172.21.15.102:0/2020745417 wait complete. 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 -- start start 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 Processor -- start 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef268db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 0x5639ef2d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef2d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639ef271240 con 0x5639ef25efa0 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5639ef270b70 con 0x5639ef259440 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639ef271910 con 0x5639ef259e30 2024-04-24T23:12:27.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f2064577740 10 client.0 _ll_register_callbacks cb 0x7f204c092da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f205b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef2d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f205b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef2d4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60832/0 (socket says 172.21.15.102:60832) 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.783+0000 7f205b7fe640 1 -- 172.21.15.102:0/2905094068 learned_addr learned my addr 172.21.15.102:0/2905094068 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205a7fc640 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 0x5639ef2d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205affd640 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef268db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205b7fe640 1 -- 172.21.15.102:0/2905094068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 msgr2=0x5639ef2d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205b7fe640 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 0x5639ef2d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205b7fe640 1 -- 172.21.15.102:0/2905094068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 msgr2=0x5639ef268db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205b7fe640 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef268db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205b7fe640 1 -- 172.21.15.102:0/2905094068 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ef2d4bc0 con 0x5639ef25efa0 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205a7fc640 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 0x5639ef2d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205affd640 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef268db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f205b7fe640 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef2d4430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2054031d80 tx=0x7f2054031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193ad4dce5eefa6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f203affd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:27.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f203affd640 1 -- 172.21.15.102:0/2905094068 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205404e050 con 0x5639ef25efa0 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f203affd640 1 -- 172.21.15.102:0/2905094068 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f2054004d40 con 0x5639ef25efa0 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f2064577740 1 -- 172.21.15.102:0/2905094068 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20540044e0 con 0x5639ef25efa0 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f203affd640 1 -- 172.21.15.102:0/2905094068 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2054004340 con 0x5639ef25efa0 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f2064577740 10 client.15084 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f2064577740 1 -- 172.21.15.102:0/2905094068 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5639ef2d5e30 con 0x5639ef25efa0 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f203affd640 1 -- 172.21.15.102:0/2905094068 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20540044e0 con 0x5639ef25efa0 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f203affd640 1 -- 172.21.15.102:0/2905094068 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f205405a030 con 0x5639ef25efa0 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f2064577740 10 client.15084 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f2064577740 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.784+0000 7f2064577740 20 client.15084 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f2064577740 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x5639ef2da780 0x5639ef2d9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f2064577740 4 client.15084 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f2064577740 1 -- 172.21.15.102:0/2905094068 --> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] -- command(tid 0: {"prefix": "status"}) -- 0x5639ef1c4790 con 0x5639ef2da780 2024-04-24T23:12:27.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f205affd640 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x5639ef2da780 0x5639ef2d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f205affd640 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x5639ef2da780 0x5639ef2d9490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=895e052551d1c424 server_cookie=a5a8a964b1721baa in_seq=0 out_seq=0 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f203affd640 10 client.15084 ms_handle_connect on v2:172.21.15.161:6836/2386140374 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f203affd640 1 -- 172.21.15.102:0/2905094068 <== mds.1 v2:172.21.15.161:6836/2386140374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5639ef1c4790 con 0x5639ef2da780 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f203affd640 10 client.15084 handle_command_reply: tid=0 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f2064577740 1 client.15084 shutdown 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f2064577740 1 -- 172.21.15.102:0/2905094068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 msgr2=0x5639ef2d4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f2064577740 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef2d4430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2054031d80 tx=0x7f2054031bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f2064577740 1 -- 172.21.15.102:0/2905094068 shutdown_connections 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f2064577740 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x5639ef2da780 0x5639ef2d9490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.785+0000 7f2064577740 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639ef25efa0 0x5639ef2d4430 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f2064577740 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639ef259e30 0x5639ef2d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f203affd640 0 client.15084 ms_handle_reset on v2:172.21.15.161:6836/2386140374 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f2064577740 1 --2- 172.21.15.102:0/2905094068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639ef259440 0x5639ef268db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f2064577740 1 -- 172.21.15.102:0/2905094068 >> 172.21.15.102:0/2905094068 conn(0x5639ef254ab0 msgr2=0x5639ef24f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f2064577740 1 -- 172.21.15.102:0/2905094068 shutdown_connections 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f2064577740 1 -- 172.21.15.102:0/2905094068 wait complete. 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f2064577740 20 client.15084 trim_cache size 0 max 16384 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 1, 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 24473, 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 84.463590260999993, 2024-04-24T23:12:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 20, 2024-04-24T23:12:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 88, 2024-04-24T23:12:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T23:12:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 103.77320238999999 2024-04-24T23:12:27.797 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:12:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 -- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f20440a2b90 msgr2=0x7f20440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f20440a2b90 0x7f20440a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2054031aa0 tx=0x7f2054031530 comp rx=0 tx=0).stop 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 -- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c101880 msgr2=0x7f205c19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c101880 0x7f205c19a130 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f204c00d950 tx=0x7f204c00de10 comp rx=0 tx=0).stop 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 -- 172.21.15.102:0/3083008278 shutdown_connections 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f20440a2b90 0x7f20440a5050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c102240 0x7f205c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f205c101880 0x7f205c19a130 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 --2- 172.21.15.102:0/3083008278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f205c100690 0x7f205c199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 -- 172.21.15.102:0/3083008278 >> 172.21.15.102:0/3083008278 conn(0x7f205c0fbc00 msgr2=0x7f205c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 -- 172.21.15.102:0/3083008278 shutdown_connections 2024-04-24T23:12:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.786+0000 7f206186d640 1 -- 172.21.15.102:0/3083008278 wait complete. 2024-04-24T23:12:27.799 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:12:27.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.901+0000 7ff124110640 1 -- 172.21.15.102:0/4237757357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c102990 msgr2=0x7ff11c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.901+0000 7ff124110640 1 --2- 172.21.15.102:0/4237757357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c102990 0x7ff11c102e10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff118009900 tx=0x7ff11802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:27.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.901+0000 7ff124110640 1 -- 172.21.15.102:0/4237757357 shutdown_connections 2024-04-24T23:12:27.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.901+0000 7ff124110640 1 --2- 172.21.15.102:0/4237757357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff11c103350 0x7ff11c109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.901+0000 7ff124110640 1 --2- 172.21.15.102:0/4237757357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c102990 0x7ff11c102e10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.901+0000 7ff124110640 1 --2- 172.21.15.102:0/4237757357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff11c1017a0 0x7ff11c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.901+0000 7ff124110640 1 -- 172.21.15.102:0/4237757357 >> 172.21.15.102:0/4237757357 conn(0x7ff11c0fcd10 msgr2=0x7ff11c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 -- 172.21.15.102:0/4237757357 shutdown_connections 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 -- 172.21.15.102:0/4237757357 wait complete. 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 Processor -- start 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 -- start start 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff11c1017a0 0x7ff11c1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff11c102990 0x7ff11c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c103350 0x7ff11c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c10d300 con 0x7ff11c103350 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c10d180 con 0x7ff11c102990 2024-04-24T23:12:27.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff124110640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c10d480 con 0x7ff11c1017a0 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff1226ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c103350 0x7ff11c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff1226ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c103350 0x7ff11c19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60844/0 (socket says 172.21.15.102:60844) 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff1226ad640 1 -- 172.21.15.102:0/3932300675 learned_addr learned my addr 172.21.15.102:0/3932300675 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.902+0000 7ff121eac640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff11c1017a0 0x7ff11c1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff1216ab640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff11c102990 0x7ff11c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff1226ad640 1 -- 172.21.15.102:0/3932300675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff11c1017a0 msgr2=0x7ff11c1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff1226ad640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff11c1017a0 0x7ff11c1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff1226ad640 1 -- 172.21.15.102:0/3932300675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff11c102990 msgr2=0x7ff11c197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff1226ad640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff11c102990 0x7ff11c197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff1226ad640 1 -- 172.21.15.102:0/3932300675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11c19f5c0 con 0x7ff11c103350 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff1216ab640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff11c102990 0x7ff11c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff1226ad640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c103350 0x7ff11c19ef50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff11400cce0 tx=0x7ff11400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff112ffd640 1 -- 172.21.15.102:0/3932300675 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11400f450 con 0x7ff11c103350 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff112ffd640 1 -- 172.21.15.102:0/3932300675 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff11400faa0 con 0x7ff11c103350 2024-04-24T23:12:27.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff124110640 1 -- 172.21.15.102:0/3932300675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11c19f8b0 con 0x7ff11c103350 2024-04-24T23:12:27.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff112ffd640 1 -- 172.21.15.102:0/3932300675 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114018b70 con 0x7ff11c103350 2024-04-24T23:12:27.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.903+0000 7ff124110640 1 -- 172.21.15.102:0/3932300675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11c19fdb0 con 0x7ff11c103350 2024-04-24T23:12:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.905+0000 7ff112ffd640 1 -- 172.21.15.102:0/3932300675 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff114018d10 con 0x7ff11c103350 2024-04-24T23:12:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.905+0000 7ff126e1e740 1 Processor -- start 2024-04-24T23:12:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff126e1e740 1 -- start start 2024-04-24T23:12:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff112ffd640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1080a2b90 0x7ff1080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 0x563a1b294860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b299930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b299fa0 0x563a1b29c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff126e1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563a1b1f7f00 con 0x563a1b294e30 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff126e1e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a1b1a0750 con 0x563a1b299fa0 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff126e1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a1b1f3ec0 con 0x563a1b294440 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff112ffd640 1 -- 172.21.15.102:0/3932300675 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff1140cf4e0 con 0x7ff11c103350 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff1216ab640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1080a2b90 0x7ff1080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff1216ab640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b299930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff121eac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 0x563a1b294860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff1216ab640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b299930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60850/0 (socket says 172.21.15.102:60850) 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff1216ab640 1 -- 172.21.15.102:0/3533338559 learned_addr learned my addr 172.21.15.102:0/3533338559 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff1226ad640 1 --2- 172.21.15.102:0/3533338559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b299fa0 0x563a1b29c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff1216ab640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1080a2b90 0x7ff1080a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff118033000 tx=0x7ff118000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff121eac640 1 -- 172.21.15.102:0/3533338559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b299fa0 msgr2=0x563a1b29c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff121eac640 1 --2- 172.21.15.102:0/3533338559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b299fa0 0x563a1b29c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff121eac640 1 -- 172.21.15.102:0/3533338559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 msgr2=0x563a1b299930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff121eac640 1 --2- 172.21.15.102:0/3533338559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b299930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff121eac640 1 -- 172.21.15.102:0/3533338559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a1af0c2d0 con 0x563a1b294440 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.906+0000 7ff1216ab640 1 --2- 172.21.15.102:0/3533338559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b299930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.907+0000 7ff121eac640 1 --2- 172.21.15.102:0/3533338559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 0x563a1b294860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00ab20 tx=0x7ff10c00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f56ab17589aa3a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.907+0000 7ff107fff640 1 -- 172.21.15.102:0/3533338559 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c004020 con 0x563a1b294440 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.907+0000 7ff107fff640 1 -- 172.21.15.102:0/3533338559 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff10c004c50 con 0x563a1b294440 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.907+0000 7ff107fff640 1 -- 172.21.15.102:0/3533338559 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c005460 con 0x563a1b294440 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.907+0000 7ff126e1e740 1 -- 172.21.15.102:0/3533338559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 msgr2=0x563a1b294860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.907+0000 7ff126e1e740 1 --2- 172.21.15.102:0/3533338559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 0x563a1b294860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00ab20 tx=0x7ff10c00f3c0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 -- 172.21.15.102:0/3533338559 shutdown_connections 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 --2- 172.21.15.102:0/3533338559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b299fa0 0x563a1b29c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 --2- 172.21.15.102:0/3533338559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b299930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 --2- 172.21.15.102:0/3533338559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 0x563a1b294860 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 -- 172.21.15.102:0/3533338559 >> 172.21.15.102:0/3533338559 conn(0x563a1b28fab0 msgr2=0x563a1b28feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 -- 172.21.15.102:0/3533338559 shutdown_connections 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 -- 172.21.15.102:0/3533338559 wait complete. 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 -- start start 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 Processor -- start 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 0x563a1b29e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294e30 0x563a1b29ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563a1b29cf50 con 0x563a1b299fa0 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a1b1b3cf0 con 0x563a1b294440 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a1b29d0d0 con 0x563a1b294e30 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.908+0000 7ff126e1e740 10 client.0 _ll_register_callbacks cb 0x563a1b1b36e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1226ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1226ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60852/0 (socket says 172.21.15.102:60852) 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1226ad640 1 -- 172.21.15.102:0/1835648430 learned_addr learned my addr 172.21.15.102:0/1835648430 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff121eac640 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 0x563a1b29e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1216ab640 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294e30 0x563a1b29ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1226ad640 1 -- 172.21.15.102:0/1835648430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294e30 msgr2=0x563a1b29ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1226ad640 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294e30 0x563a1b29ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1226ad640 1 -- 172.21.15.102:0/1835648430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 msgr2=0x563a1b29e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1226ad640 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 0x563a1b29e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1226ad640 1 -- 172.21.15.102:0/1835648430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a1afd66b0 con 0x563a1b299fa0 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1226ad640 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2a5d90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff11409bcc0 tx=0x7ff11400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf57072f77ec031 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1057fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1057fa640 1 -- 172.21.15.102:0/1835648430 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1140cf2e0 con 0x563a1b299fa0 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1057fa640 1 -- 172.21.15.102:0/1835648430 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff1140a4510 con 0x563a1b299fa0 2024-04-24T23:12:27.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff126e1e740 1 -- 172.21.15.102:0/1835648430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff10c004c50 con 0x563a1b299fa0 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1057fa640 1 -- 172.21.15.102:0/1835648430 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff1140a49a0 con 0x563a1b299fa0 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff126e1e740 10 client.15096 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff126e1e740 1 -- 172.21.15.102:0/1835648430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563a1b2a8e40 con 0x563a1b299fa0 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.909+0000 7ff1057fa640 1 -- 172.21.15.102:0/1835648430 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff10c004c50 con 0x563a1b299fa0 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff1057fa640 1 -- 172.21.15.102:0/1835648430 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7ff1140b2070 con 0x563a1b299fa0 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff126e1e740 10 client.15096 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff126e1e740 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff126e1e740 20 client.15096 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff126e1e740 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563a1b2ab510 0x563a1b2ac980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff126e1e740 4 client.15096 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff126e1e740 1 -- 172.21.15.102:0/1835648430 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a1b205440 con 0x563a1b2ab510 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff121eac640 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563a1b2ab510 0x563a1b2ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff1057fa640 10 client.15096 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.910+0000 7ff121eac640 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563a1b2ab510 0x563a1b2ac980 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291be163a665beba server_cookie=e3a773e9febbfb4b in_seq=0 out_seq=0 2024-04-24T23:12:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff1057fa640 1 -- 172.21.15.102:0/1835648430 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a1b205440 con 0x563a1b2ab510 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff1057fa640 10 client.15096 handle_command_reply: tid=0 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 client.15096 shutdown 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 -- 172.21.15.102:0/1835648430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 msgr2=0x563a1b2a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2a5d90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff11409bcc0 tx=0x7ff11400fd10 comp rx=0 tx=0).stop 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 -- 172.21.15.102:0/1835648430 shutdown_connections 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563a1b2ab510 0x563a1b2ac980 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2a5d90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294e30 0x563a1b29ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff1057fa640 0 client.15096 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 --2- 172.21.15.102:0/1835648430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 0x563a1b29e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 -- 172.21.15.102:0/1835648430 >> 172.21.15.102:0/1835648430 conn(0x563a1b28fab0 msgr2=0x563a1b28eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 -- 172.21.15.102:0/1835648430 shutdown_connections 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 1 -- 172.21.15.102:0/1835648430 wait complete. 2024-04-24T23:12:27.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.911+0000 7ff126e1e740 20 client.15096 trim_cache size 0 max 16384 2024-04-24T23:12:27.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff126e1e740 1 Processor -- start 2024-04-24T23:12:27.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff126e1e740 1 -- start start 2024-04-24T23:12:27.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 0x563a1b2a6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294e30 0x563a1b2a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff126e1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563a1b2aacc0 con 0x563a1b299fa0 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff126e1e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a1b2ab390 con 0x563a1b294e30 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff126e1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a1b2aba60 con 0x563a1b294440 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff1226ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff1226ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60864/0 (socket says 172.21.15.102:60864) 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff1226ad640 1 -- 172.21.15.102:0/209851339 learned_addr learned my addr 172.21.15.102:0/209851339 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff1226ad640 1 -- 172.21.15.102:0/209851339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 msgr2=0x563a1b2a6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff1226ad640 1 --2- 172.21.15.102:0/209851339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 0x563a1b2a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff1226ad640 1 -- 172.21.15.102:0/209851339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294e30 msgr2=0x563a1b2a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff1226ad640 1 --2- 172.21.15.102:0/209851339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294e30 0x563a1b2a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff1226ad640 1 -- 172.21.15.102:0/209851339 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a1b299440 con 0x563a1b299fa0 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.929+0000 7ff1226ad640 1 --2- 172.21.15.102:0/209851339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2961f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff114099fe0 tx=0x7ff11409a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23def01d7351ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff1077fe640 1 -- 172.21.15.102:0/209851339 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1140a4510 con 0x563a1b299fa0 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff1077fe640 1 -- 172.21.15.102:0/209851339 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1140a46b0 con 0x563a1b299fa0 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff1077fe640 1 -- 172.21.15.102:0/209851339 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1140cf330 con 0x563a1b299fa0 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 -- 172.21.15.102:0/209851339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 msgr2=0x563a1b2961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 --2- 172.21.15.102:0/209851339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2961f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff114099fe0 tx=0x7ff11409a240 comp rx=0 tx=0).stop 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 -- 172.21.15.102:0/209851339 shutdown_connections 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 --2- 172.21.15.102:0/209851339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b299fa0 0x563a1b2961f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 --2- 172.21.15.102:0/209851339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294e30 0x563a1b2a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 --2- 172.21.15.102:0/209851339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b294440 0x563a1b2a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 -- 172.21.15.102:0/209851339 >> 172.21.15.102:0/209851339 conn(0x563a1b28fab0 msgr2=0x563a1b28a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 -- 172.21.15.102:0/209851339 shutdown_connections 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 -- 172.21.15.102:0/209851339 wait complete. 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.930+0000 7ff126e1e740 1 -- start start 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff126e1e740 1 Processor -- start 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 0x563a1b2a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b30d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff126e1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b299fa0 0x563a1b30fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff126e1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563a1b2ac800 con 0x563a1b294e30 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff126e1e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a1b2ac130 con 0x563a1b294440 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff126e1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a1b2aced0 con 0x563a1b299fa0 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff126e1e740 10 client.0 _ll_register_callbacks cb 0x7ff11409b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1216ab640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b30d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1216ab640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b30d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60868/0 (socket says 172.21.15.102:60868) 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1216ab640 1 -- 172.21.15.102:0/2328993715 learned_addr learned my addr 172.21.15.102:0/2328993715 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1226ad640 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b299fa0 0x563a1b30fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff121eac640 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 0x563a1b2a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1216ab640 1 -- 172.21.15.102:0/2328993715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b299fa0 msgr2=0x563a1b30fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1216ab640 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b299fa0 0x563a1b30fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1216ab640 1 -- 172.21.15.102:0/2328993715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 msgr2=0x563a1b2a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1216ab640 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 0x563a1b2a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1216ab640 1 -- 172.21.15.102:0/2328993715 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a1b310310 con 0x563a1b294e30 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1216ab640 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b30d4a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff118005ab0 tx=0x7ff118009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4244aa024ad97526 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.931+0000 7ff1057fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff1057fa640 1 -- 172.21.15.102:0/2328993715 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118035050 con 0x563a1b294e30 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff1057fa640 1 -- 172.21.15.102:0/2328993715 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff1180035f0 con 0x563a1b294e30 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff126e1e740 1 -- 172.21.15.102:0/2328993715 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1140a46b0 con 0x563a1b294e30 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff1057fa640 1 -- 172.21.15.102:0/2328993715 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff1180037f0 con 0x563a1b294e30 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff126e1e740 10 client.15102 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff126e1e740 1 -- 172.21.15.102:0/2328993715 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563a1b311610 con 0x563a1b294e30 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff1057fa640 1 -- 172.21.15.102:0/2328993715 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1140a46b0 con 0x563a1b294e30 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff1057fa640 1 -- 172.21.15.102:0/2328993715 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7ff1180329f0 con 0x563a1b294e30 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff126e1e740 10 client.15102 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff126e1e740 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:27.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff126e1e740 20 client.15102 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff126e1e740 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563a1b316200 0x563a1b312d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff126e1e740 4 client.15102 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff126e1e740 1 -- 172.21.15.102:0/2328993715 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "status"}) -- 0x563a1b1ff790 con 0x563a1b316200 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.932+0000 7ff121eac640 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563a1b316200 0x563a1b312d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff1057fa640 10 client.15102 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff121eac640 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563a1b316200 0x563a1b312d10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6a93e0ec7ba142 server_cookie=170ef3ca51137157 in_seq=0 out_seq=0 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff1057fa640 1 -- 172.21.15.102:0/2328993715 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563a1b1ff790 con 0x563a1b316200 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff1057fa640 10 client.15102 handle_command_reply: tid=0 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 client.15102 shutdown 2024-04-24T23:12:27.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 -- 172.21.15.102:0/2328993715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 msgr2=0x563a1b30d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b30d4a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff118005ab0 tx=0x7ff118009260 comp rx=0 tx=0).stop 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 -- 172.21.15.102:0/2328993715 shutdown_connections 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1b299fa0 0x563a1b30fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563a1b316200 0x563a1b312d10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a1b294e30 0x563a1b30d4a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff1057fa640 0 client.15102 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 --2- 172.21.15.102:0/2328993715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a1b294440 0x563a1b2a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 -- 172.21.15.102:0/2328993715 >> 172.21.15.102:0/2328993715 conn(0x563a1b28fab0 msgr2=0x563a1b28a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 -- 172.21.15.102:0/2328993715 shutdown_connections 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 1 -- 172.21.15.102:0/2328993715 wait complete. 2024-04-24T23:12:27.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.933+0000 7ff126e1e740 20 client.15102 trim_cache size 0 max 16384 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 0, 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 24497, 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 86.642267717999999, 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 20, 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 88, 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 92.406268195999999 2024-04-24T23:12:27.945 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 -- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1080a2b90 msgr2=0x7ff1080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1080a2b90 0x7ff1080a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff118033000 tx=0x7ff118000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 -- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c103350 msgr2=0x7ff11c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c103350 0x7ff11c19ef50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff11400cce0 tx=0x7ff11400a620 comp rx=0 tx=0).stop 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 -- 172.21.15.102:0/3932300675 shutdown_connections 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff1080a2b90 0x7ff1080a5050 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff11c103350 0x7ff11c19ef50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff11c102990 0x7ff11c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 --2- 172.21.15.102:0/3932300675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff11c1017a0 0x7ff11c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 -- 172.21.15.102:0/3932300675 >> 172.21.15.102:0/3932300675 conn(0x7ff11c0fcd10 msgr2=0x7ff11c105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 -- 172.21.15.102:0/3932300675 shutdown_connections 2024-04-24T23:12:27.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:27.934+0000 7ff124110640 1 -- 172.21.15.102:0/3932300675 wait complete. 2024-04-24T23:12:27.948 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T23:12:27.948 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:12:28.847 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:12:28.847 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T23:12:28.848 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:28 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3599798665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:28.849 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:28 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3599798665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:28.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:28 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3599798665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:29.095 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:29.483 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:29 smithi102 ceph-mon[29216]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-24T23:12:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:29 smithi161 ceph-mon[32709]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-24T23:12:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:29 smithi183 ceph-mon[33137]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-24T23:12:30.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 -- 172.21.15.102:0/1117769502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754101190 msgr2=0x7fd7541015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/1117769502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754101190 0x7fd7541015b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd7440099c0 tx=0x7fd744009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 -- 172.21.15.102:0/1117769502 shutdown_connections 2024-04-24T23:12:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/1117769502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd754102d30 0x7fd7541095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/1117769502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754101190 0x7fd7541015b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/1117769502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd754102390 0x7fd7541027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 -- 172.21.15.102:0/1117769502 >> 172.21.15.102:0/1117769502 conn(0x7fd7540fc730 msgr2=0x7fd7540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 -- 172.21.15.102:0/1117769502 shutdown_connections 2024-04-24T23:12:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 -- 172.21.15.102:0/1117769502 wait complete. 2024-04-24T23:12:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 Processor -- start 2024-04-24T23:12:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.906+0000 7fd75a2e2700 1 -- start start 2024-04-24T23:12:30.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd75a2e2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd754101190 0x7fd754198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd75a2e2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754102390 0x7fd754198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd75a2e2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd754102d30 0x7fd75419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd75a2e2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd75410cc00 con 0x7fd754102390 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd75a2e2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd75410ca80 con 0x7fd754101190 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd75a2e2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd75410cd80 con 0x7fd754102d30 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7537fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754102390 0x7fd754198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7537fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754102390 0x7fd754198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60896/0 (socket says 172.21.15.102:60896) 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7537fe700 1 -- 172.21.15.102:0/3389978362 learned_addr learned my addr 172.21.15.102:0/3389978362 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd753fff700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd754101190 0x7fd754198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd75887f700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd754102d30 0x7fd75419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7537fe700 1 -- 172.21.15.102:0/3389978362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd754102d30 msgr2=0x7fd75419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7537fe700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd754102d30 0x7fd75419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7537fe700 1 -- 172.21.15.102:0/3389978362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd754101190 msgr2=0x7fd754198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:30.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7537fe700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd754101190 0x7fd754198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:30.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7537fe700 1 -- 172.21.15.102:0/3389978362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7541a0070 con 0x7fd754102390 2024-04-24T23:12:30.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7537fe700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754102390 0x7fd754198950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd74800c950 tx=0x7fd74800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7517fa700 1 -- 172.21.15.102:0/3389978362 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748007ac0 con 0x7fd754102390 2024-04-24T23:12:30.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd7517fa700 1 -- 172.21.15.102:0/3389978362 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd748004ce0 con 0x7fd754102390 2024-04-24T23:12:30.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.907+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7541a0360 con 0x7fd754102390 2024-04-24T23:12:30.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.908+0000 7fd7517fa700 1 -- 172.21.15.102:0/3389978362 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7480055e0 con 0x7fd754102390 2024-04-24T23:12:30.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.908+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7541a08a0 con 0x7fd754102390 2024-04-24T23:12:30.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.909+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd754068f20 con 0x7fd754102390 2024-04-24T23:12:30.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.910+0000 7fd7517fa700 1 -- 172.21.15.102:0/3389978362 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd748004750 con 0x7fd754102390 2024-04-24T23:12:30.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.910+0000 7fd7517fa700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd73c0a2c20 0x7fd73c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:30.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.910+0000 7fd753fff700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd73c0a2c20 0x7fd73c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:30.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.910+0000 7fd7517fa700 1 -- 172.21.15.102:0/3389978362 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd7480cece0 con 0x7fd754102390 2024-04-24T23:12:30.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.911+0000 7fd753fff700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd73c0a2c20 0x7fd73c0a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd744000c00 tx=0x7fd744005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:30.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:30.915+0000 7fd7517fa700 1 -- 172.21.15.102:0/3389978362 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd748014030 con 0x7fd754102390 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.131+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd75406a670 con 0x7fd73c0a2c20 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.133+0000 7fd7517fa700 1 -- 172.21.15.102:0/3389978362 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd75406a670 con 0x7fd73c0a2c20 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi102.stdout:Backend: cephadm 2024-04-24T23:12:31.135 INFO:teuthology.orchestra.run.smithi102.stdout:Available: Yes 2024-04-24T23:12:31.135 INFO:teuthology.orchestra.run.smithi102.stdout:Paused: No 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd73c0a2c20 msgr2=0x7fd73c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd73c0a2c20 0x7fd73c0a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd744000c00 tx=0x7fd744005fd0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754102390 msgr2=0x7fd754198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754102390 0x7fd754198950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd74800c950 tx=0x7fd74800cd10 comp rx=0 tx=0).stop 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 shutdown_connections 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd754102d30 0x7fd75419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd754101190 0x7fd754198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fd73c0a2c20 0x7fd73c0a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 --2- 172.21.15.102:0/3389978362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd754102390 0x7fd754198950 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 >> 172.21.15.102:0/3389978362 conn(0x7fd7540fc730 msgr2=0x7fd754069f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 shutdown_connections 2024-04-24T23:12:31.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:31.135+0000 7fd75a2e2700 1 -- 172.21.15.102:0/3389978362 wait complete. 2024-04-24T23:12:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:31 smithi161 ceph-mon[32709]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T23:12:31.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:31 smithi183 ceph-mon[33137]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T23:12:31.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:31 smithi102 ceph-mon[29216]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T23:12:31.936 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T23:12:32.174 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:32.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:32 smithi102 ceph-mon[29216]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:32.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:32 smithi161 ceph-mon[32709]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:32.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:32 smithi183 ceph-mon[33137]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:33.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:33 smithi102 ceph-mon[29216]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-24T23:12:33.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:33 smithi183 ceph-mon[33137]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 -- 172.21.15.102:0/1802142789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70102390 msgr2=0x7fbd701027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/1802142789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70102390 0x7fbd701027f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbd640099c0 tx=0x7fbd64009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 -- 172.21.15.102:0/1802142789 shutdown_connections 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/1802142789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70102d30 0x7fbd701095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/1802142789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd70101190 0x7fbd701015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/1802142789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70102390 0x7fbd701027f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 -- 172.21.15.102:0/1802142789 >> 172.21.15.102:0/1802142789 conn(0x7fbd700fc750 msgr2=0x7fbd700feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 -- 172.21.15.102:0/1802142789 shutdown_connections 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 -- 172.21.15.102:0/1802142789 wait complete. 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.845+0000 7fbd74c4e700 1 Processor -- start 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd74c4e700 1 -- start start 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd74c4e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70101190 0x7fbd7006b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd74c4e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70102390 0x7fbd7019d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd74c4e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd70102d30 0x7fbd7019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd74c4e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7010ccb0 con 0x7fbd70101190 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd74c4e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7010cb30 con 0x7fbd70102390 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd74c4e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7010ce30 con 0x7fbd70102d30 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70101190 0x7fbd7006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70101190 0x7fbd7006b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60906/0 (socket says 172.21.15.102:60906) 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6e59c700 1 -- 172.21.15.102:0/2259705373 learned_addr learned my addr 172.21.15.102:0/2259705373 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6dd9b700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70102390 0x7fbd7019d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6ed9d700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd70102d30 0x7fbd7019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6e59c700 1 -- 172.21.15.102:0/2259705373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd70102d30 msgr2=0x7fbd7019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6e59c700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd70102d30 0x7fbd7019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6e59c700 1 -- 172.21.15.102:0/2259705373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70102390 msgr2=0x7fbd7019d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6e59c700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70102390 0x7fbd7019d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6e59c700 1 -- 172.21.15.102:0/2259705373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd701a0090 con 0x7fbd70101190 2024-04-24T23:12:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6ed9d700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd70102d30 0x7fbd7019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.846+0000 7fbd6e59c700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70101190 0x7fbd7006b500 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd68007f10 tx=0x7fbd6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.847+0000 7fbd5f7fe700 1 -- 172.21.15.102:0/2259705373 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800ddd0 con 0x7fbd70101190 2024-04-24T23:12:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.847+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd701a0380 con 0x7fbd70101190 2024-04-24T23:12:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.847+0000 7fbd5f7fe700 1 -- 172.21.15.102:0/2259705373 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd68018070 con 0x7fbd70101190 2024-04-24T23:12:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.847+0000 7fbd5f7fe700 1 -- 172.21.15.102:0/2259705373 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680125b0 con 0x7fbd70101190 2024-04-24T23:12:33.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.847+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd701a0940 con 0x7fbd70101190 2024-04-24T23:12:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.848+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd70068f20 con 0x7fbd70101190 2024-04-24T23:12:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.849+0000 7fbd5f7fe700 1 -- 172.21.15.102:0/2259705373 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbd680133f0 con 0x7fbd70101190 2024-04-24T23:12:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.849+0000 7fbd5f7fe700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbd580a2c20 0x7fbd580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.849+0000 7fbd6dd9b700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbd580a2c20 0x7fbd580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.850+0000 7fbd5f7fe700 1 -- 172.21.15.102:0/2259705373 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fbd680ceed0 con 0x7fbd70101190 2024-04-24T23:12:33.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.850+0000 7fbd6dd9b700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbd580a2c20 0x7fbd580a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd7006ba80 tx=0x7fbd64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:33.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:33.855+0000 7fbd5f7fe700 1 -- 172.21.15.102:0/2259705373 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbd68010070 con 0x7fbd70101190 2024-04-24T23:12:33.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:33 smithi161 ceph-mon[32709]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-24T23:12:34.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.065+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbd70062670 con 0x7fbd580a2c20 2024-04-24T23:12:34.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.079+0000 7fbd5f7fe700 1 -- 172.21.15.102:0/2259705373 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fbd70062670 con 0x7fbd580a2c20 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.a smithi102 running (101s) 19s ago 101s 15.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 330491a062d5 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.b smithi161 running (98s) 32s ago 98s 25.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3182ad60e617 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.c smithi183 running (113s) 32s ago 113s 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef f2e55b8c423e 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.d smithi102 running (2m) 19s ago 2m 18.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 2dcddcfd0cb4 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.e smithi161 running (119s) 32s ago 118s 19.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 623b1dcc0383 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.f smithi183 running (116s) 32s ago 116s 18.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 57a123290fd9 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.g smithi102 running (104s) 19s ago 104s 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3f1a557f2ec0 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.h smithi161 running (110s) 32s ago 109s 20.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 0edfb28a372e 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.i smithi183 running (107s) 32s ago 107s 17.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef a1bcedce8d3f 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.j smithi102 running (2m) 19s ago 2m 15.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 4ae03cbd359f 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.k smithi161 running (2m) 32s ago 2m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 15bbc090efde 2024-04-24T23:12:34.081 INFO:teuthology.orchestra.run.smithi102.stdout:mds.l smithi183 running (2m) 32s ago 2m 13.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a89704907663 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.x smithi102 *:9283,8765 running (9m) 19s ago 9m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 78ac8b135b5e 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.y smithi161 *:8443,8765 running (7m) 32s ago 7m 454M - 19.0.0-2445-gd2c45d97 f68e37b807ef 0d561e359138 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.z smithi183 *:8443,8765 running (7m) 32s ago 7m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef a929bbfac236 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:mon.a smithi102 running (9m) 19s ago 9m 50.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 4c8285e9b32c 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:mon.b smithi161 running (7m) 32s ago 7m 38.8M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef b2b0f3439b1c 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:mon.c smithi183 running (8m) 32s ago 8m 39.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef daac70bc659d 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (7m) 19s ago 7m 47.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 45a701edf386 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi161 running (6m) 32s ago 6m 56.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef b818a7a900ac 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi183 running (6m) 32s ago 6m 56.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 3cc0b6e0952c 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi102 running (6m) 19s ago 6m 43.5M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 1c4e18cdd3c9 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi161 running (5m) 32s ago 5m 44.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 102325cc5933 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi183 running (5m) 32s ago 5m 56.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 8b7bc6e57982 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:osd.6 smithi102 running (4m) 19s ago 4m 42.1M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 019ffb9edd4f 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:osd.7 smithi161 running (4m) 32s ago 4m 42.5M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 36b262d881b1 2024-04-24T23:12:34.082 INFO:teuthology.orchestra.run.smithi102.stdout:osd.8 smithi183 running (3m) 32s ago 3m 42.3M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 3b3dd2989d75 2024-04-24T23:12:34.083 INFO:teuthology.orchestra.run.smithi102.stdout:osd.9 smithi102 running (3m) 19s ago 3m 50.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef ade063394ff4 2024-04-24T23:12:34.083 INFO:teuthology.orchestra.run.smithi102.stdout:osd.10 smithi161 running (3m) 32s ago 3m 43.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef fa73740dec10 2024-04-24T23:12:34.083 INFO:teuthology.orchestra.run.smithi102.stdout:osd.11 smithi183 running (2m) 32s ago 2m 54.5M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef a36b90069a5d 2024-04-24T23:12:34.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbd580a2c20 msgr2=0x7fbd580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbd580a2c20 0x7fbd580a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd7006ba80 tx=0x7fbd64005fb0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70101190 msgr2=0x7fbd7006b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70101190 0x7fbd7006b500 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd68007f10 tx=0x7fbd6800d3b0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 shutdown_connections 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd70102d30 0x7fbd7019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fbd580a2c20 0x7fbd580a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd70101190 0x7fbd7006b500 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 --2- 172.21.15.102:0/2259705373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70102390 0x7fbd7019d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 >> 172.21.15.102:0/2259705373 conn(0x7fbd700fc750 msgr2=0x7fbd70105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 shutdown_connections 2024-04-24T23:12:34.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:34.082+0000 7fbd74c4e700 1 -- 172.21.15.102:0/2259705373 wait complete. 2024-04-24T23:12:34.855 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T23:12:35.093 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:35.540 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:35 smithi102 ceph-mon[29216]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:35.540 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:35 smithi102 ceph-mon[29216]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:12:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:35 smithi183 ceph-mon[33137]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:35 smithi183 ceph-mon[33137]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:12:35.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:35 smithi161 ceph-mon[32709]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:35.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:35 smithi161 ceph-mon[32709]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:12:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 -- 172.21.15.102:0/3716632675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 msgr2=0x7f0518103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 --2- 172.21.15.102:0/3716632675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 0x7f0518103900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f05080099c0 tx=0x7f0508009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 -- 172.21.15.102:0/3716632675 shutdown_connections 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 --2- 172.21.15.102:0/3716632675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0518103e40 0x7f051810a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 --2- 172.21.15.102:0/3716632675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05181022a0 0x7f05181026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 --2- 172.21.15.102:0/3716632675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 0x7f0518103900 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 -- 172.21.15.102:0/3716632675 >> 172.21.15.102:0/3716632675 conn(0x7f05180fd840 msgr2=0x7f05180ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 -- 172.21.15.102:0/3716632675 shutdown_connections 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 -- 172.21.15.102:0/3716632675 wait complete. 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 Processor -- start 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 -- start start 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.774+0000 7f05207e6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05181022a0 0x7f0518075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f05207e6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 0x7f0518074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f05207e6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0518103e40 0x7f0518074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f05207e6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f051810ddb0 con 0x7f05181034a0 2024-04-24T23:12:36.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f05207e6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f051810dc30 con 0x7f0518103e40 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f05207e6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f051810df30 con 0x7f05181022a0 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051dd81700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 0x7f0518074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051dd81700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 0x7f0518074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56322/0 (socket says 172.21.15.102:56322) 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051dd81700 1 -- 172.21.15.102:0/3403173090 learned_addr learned my addr 172.21.15.102:0/3403173090 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051e582700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05181022a0 0x7f0518075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051ed83700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0518103e40 0x7f0518074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051dd81700 1 -- 172.21.15.102:0/3403173090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05181022a0 msgr2=0x7f0518075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051dd81700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05181022a0 0x7f0518075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051dd81700 1 -- 172.21.15.102:0/3403173090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0518103e40 msgr2=0x7f0518074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051dd81700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0518103e40 0x7f0518074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051dd81700 1 -- 172.21.15.102:0/3403173090 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0518075410 con 0x7f05181034a0 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f051dd81700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 0x7f0518074620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0508006010 tx=0x7f05080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f050f7fe700 1 -- 172.21.15.102:0/3403173090 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508027070 con 0x7f05181034a0 2024-04-24T23:12:36.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f050f7fe700 1 -- 172.21.15.102:0/3403173090 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0508022470 con 0x7f05181034a0 2024-04-24T23:12:36.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05181a24d0 con 0x7f05181034a0 2024-04-24T23:12:36.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f050f7fe700 1 -- 172.21.15.102:0/3403173090 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050800f6c0 con 0x7f05181034a0 2024-04-24T23:12:36.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.775+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05181a29a0 con 0x7f05181034a0 2024-04-24T23:12:36.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.776+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0518068f20 con 0x7f05181034a0 2024-04-24T23:12:36.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.777+0000 7f050f7fe700 1 -- 172.21.15.102:0/3403173090 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f050802a070 con 0x7f05181034a0 2024-04-24T23:12:36.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.778+0000 7f050f7fe700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05040a2ca0 0x7f05040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.778+0000 7f051e582700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05040a2ca0 0x7f05040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.778+0000 7f050f7fe700 1 -- 172.21.15.102:0/3403173090 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f05080d0520 con 0x7f05181034a0 2024-04-24T23:12:36.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.779+0000 7f051e582700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05040a2ca0 0x7f05040a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0518103300 tx=0x7f05100094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.782+0000 7f050f7fe700 1 -- 172.21.15.102:0/3403173090 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0508031070 con 0x7f05181034a0 2024-04-24T23:12:36.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.992+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f0518062670 con 0x7f05040a2ca0 2024-04-24T23:12:36.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:36.998+0000 7f050f7fe700 1 -- 172.21.15.102:0/3403173090 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f0518062670 con 0x7f05040a2ca0 2024-04-24T23:12:36.999 INFO:teuthology.orchestra.run.smithi102.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T23:12:36.999 INFO:teuthology.orchestra.run.smithi102.stdout:mds.all 12/12 35s ago 2m smithi102=a;smithi102=d;smithi102=g;smithi102=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-04-24T23:12:36.999 INFO:teuthology.orchestra.run.smithi102.stdout:mgr 3/3 35s ago 7m smithi102=x;smithi161=y;smithi183=z;count:3 2024-04-24T23:12:36.999 INFO:teuthology.orchestra.run.smithi102.stdout:mon 3/3 35s ago 8m smithi102:172.21.15.102=a;smithi161:172.21.15.161=b;smithi183:172.21.15.183=c;count:3 2024-04-24T23:12:36.999 INFO:teuthology.orchestra.run.smithi102.stdout:osd 12 35s ago - 2024-04-24T23:12:37.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05040a2ca0 msgr2=0x7f05040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05040a2ca0 0x7f05040a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0518103300 tx=0x7f05100094f0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 msgr2=0x7f0518074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 0x7f0518074620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0508006010 tx=0x7f05080047b0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 shutdown_connections 2024-04-24T23:12:37.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0518103e40 0x7f0518074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05181022a0 0x7f0518075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f05040a2ca0 0x7f05040a5160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 --2- 172.21.15.102:0/3403173090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05181034a0 0x7f0518074620 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 >> 172.21.15.102:0/3403173090 conn(0x7f05180fd840 msgr2=0x7f05181067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:37.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 shutdown_connections 2024-04-24T23:12:37.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:37.000+0000 7f05207e6700 1 -- 172.21.15.102:0/3403173090 wait complete. 2024-04-24T23:12:37.696 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:37 smithi102 ceph-mon[29216]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:37.723 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T23:12:37.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:37 smithi183 ceph-mon[33137]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:37.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:37 smithi161 ceph-mon[32709]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:37.961 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:38.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:38 smithi183 ceph-mon[33137]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:38.787 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:38 smithi102 ceph-mon[29216]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:38.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:38 smithi161 ceph-mon[32709]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:39.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.601+0000 7efc6da3f700 1 -- 172.21.15.102:0/917382597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 msgr2=0x7efc6810a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.601+0000 7efc6da3f700 1 --2- 172.21.15.102:0/917382597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 0x7efc6810a6f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc6000b0d0 tx=0x7efc6000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.601+0000 7efc6da3f700 1 -- 172.21.15.102:0/917382597 shutdown_connections 2024-04-24T23:12:39.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.601+0000 7efc6da3f700 1 --2- 172.21.15.102:0/917382597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 0x7efc6810a6f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.601+0000 7efc6da3f700 1 --2- 172.21.15.102:0/917382597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc681022a0 0x7efc681026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.601+0000 7efc6da3f700 1 --2- 172.21.15.102:0/917382597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc681034a0 0x7efc68103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.601+0000 7efc6da3f700 1 -- 172.21.15.102:0/917382597 >> 172.21.15.102:0/917382597 conn(0x7efc680fd840 msgr2=0x7efc680ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:39.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.601+0000 7efc6da3f700 1 -- 172.21.15.102:0/917382597 shutdown_connections 2024-04-24T23:12:39.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.601+0000 7efc6da3f700 1 -- 172.21.15.102:0/917382597 wait complete. 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc6da3f700 1 Processor -- start 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc6da3f700 1 -- start start 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc6da3f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc681022a0 0x7efc68198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc6da3f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc681034a0 0x7efc681988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc6da3f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 0x7efc6819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc6da3f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810dd30 con 0x7efc68103e40 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc6da3f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810dbb0 con 0x7efc681034a0 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc6da3f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810deb0 con 0x7efc681022a0 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc677fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 0x7efc6819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc677fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 0x7efc6819f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56344/0 (socket says 172.21.15.102:56344) 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc677fe700 1 -- 172.21.15.102:0/2108514184 learned_addr learned my addr 172.21.15.102:0/2108514184 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc667fc700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc681034a0 0x7efc681988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc66ffd700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc681022a0 0x7efc68198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc677fe700 1 -- 172.21.15.102:0/2108514184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc681022a0 msgr2=0x7efc68198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc677fe700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc681022a0 0x7efc68198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc677fe700 1 -- 172.21.15.102:0/2108514184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc681034a0 msgr2=0x7efc681988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc677fe700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc681034a0 0x7efc681988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.602+0000 7efc677fe700 1 -- 172.21.15.102:0/2108514184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6819ffd0 con 0x7efc68103e40 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.603+0000 7efc677fe700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 0x7efc6819f930 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc60004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.603+0000 7efc6ca3d700 1 -- 172.21.15.102:0/2108514184 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6001f030 con 0x7efc68103e40 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.603+0000 7efc6ca3d700 1 -- 172.21.15.102:0/2108514184 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc60003a00 con 0x7efc68103e40 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.603+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc681a0260 con 0x7efc68103e40 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.603+0000 7efc6ca3d700 1 -- 172.21.15.102:0/2108514184 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc600046a0 con 0x7efc68103e40 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.603+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc681a06b0 con 0x7efc68103e40 2024-04-24T23:12:39.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.604+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc68068f20 con 0x7efc68103e40 2024-04-24T23:12:39.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.605+0000 7efc6ca3d700 1 -- 172.21.15.102:0/2108514184 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efc60003bb0 con 0x7efc68103e40 2024-04-24T23:12:39.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.605+0000 7efc6ca3d700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efc500a2c20 0x7efc500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.606+0000 7efc66ffd700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efc500a2c20 0x7efc500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.606+0000 7efc6ca3d700 1 -- 172.21.15.102:0/2108514184 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7efc6000f020 con 0x7efc68103e40 2024-04-24T23:12:39.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.606+0000 7efc66ffd700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efc500a2c20 0x7efc500a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc58005fd0 tx=0x7efc5800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.609+0000 7efc6ca3d700 1 -- 172.21.15.102:0/2108514184 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc60029020 con 0x7efc68103e40 2024-04-24T23:12:39.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:39 smithi183 ceph-mon[33137]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:39.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.811+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7efc68062670 con 0x7efc500a2c20 2024-04-24T23:12:39.812 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:39 smithi102 ceph-mon[29216]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:39.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.812+0000 7efc6ca3d700 1 -- 172.21.15.102:0/2108514184 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7efc68062670 con 0x7efc500a2c20 2024-04-24T23:12:39.813 INFO:teuthology.orchestra.run.smithi102.stdout:HOST ADDR LABELS STATUS 2024-04-24T23:12:39.814 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102 172.21.15.102 2024-04-24T23:12:39.814 INFO:teuthology.orchestra.run.smithi102.stdout:smithi161 172.21.15.161 2024-04-24T23:12:39.814 INFO:teuthology.orchestra.run.smithi102.stdout:smithi183 172.21.15.183 2024-04-24T23:12:39.814 INFO:teuthology.orchestra.run.smithi102.stdout:3 hosts in cluster 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.814+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efc500a2c20 msgr2=0x7efc500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.814+0000 7efc6da3f700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efc500a2c20 0x7efc500a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc58005fd0 tx=0x7efc5800a560 comp rx=0 tx=0).stop 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.814+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 msgr2=0x7efc6819f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.814+0000 7efc6da3f700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 0x7efc6819f930 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc60004d50 comp rx=0 tx=0).stop 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.814+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 shutdown_connections 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.814+0000 7efc6da3f700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7efc500a2c20 0x7efc500a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.814+0000 7efc6da3f700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc68103e40 0x7efc6819f930 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.814+0000 7efc6da3f700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc681022a0 0x7efc68198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.814+0000 7efc6da3f700 1 --2- 172.21.15.102:0/2108514184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc681034a0 0x7efc681988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.815+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 >> 172.21.15.102:0/2108514184 conn(0x7efc680fd840 msgr2=0x7efc681067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:39.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.815+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 shutdown_connections 2024-04-24T23:12:39.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:39.815+0000 7efc6da3f700 1 -- 172.21.15.102:0/2108514184 wait complete. 2024-04-24T23:12:39.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:39 smithi161 ceph-mon[32709]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:40.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:40 smithi102 ceph-mon[29216]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:40.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:40 smithi102 ceph-mon[29216]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:40.627 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T23:12:40.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:40 smithi183 ceph-mon[33137]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:40.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:40 smithi183 ceph-mon[33137]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:40.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:40 smithi161 ceph-mon[32709]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:40.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:40 smithi161 ceph-mon[32709]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:12:40.866 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:42.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2292597048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84102290 msgr2=0x7f7c841026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2292597048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84102290 0x7f7c841026b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c780099c0 tx=0x7f7c78009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2292597048 shutdown_connections 2024-04-24T23:12:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2292597048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84103e30 0x7f7c8410a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2292597048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84102290 0x7f7c841026b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2292597048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c84103490 0x7f7c841038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2292597048 >> 172.21.15.102:0/2292597048 conn(0x7f7c840fd830 msgr2=0x7f7c840ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2292597048 shutdown_connections 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2292597048 wait complete. 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 Processor -- start 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 -- start start 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c84102290 0x7f7c84198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84103490 0x7f7c841989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84103e30 0x7f7c8419fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410dda0 con 0x7f7c84103490 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.723+0000 7f7c8ae81700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410dc20 con 0x7f7c84102290 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c8ae81700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410df20 con 0x7f7c84103e30 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c83fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84103490 0x7f7c841989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c83fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84103490 0x7f7c841989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56354/0 (socket says 172.21.15.102:56354) 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c83fff700 1 -- 172.21.15.102:0/2218100763 learned_addr learned my addr 172.21.15.102:0/2218100763 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c88c1d700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c84102290 0x7f7c84198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c8941e700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84103e30 0x7f7c8419fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c83fff700 1 -- 172.21.15.102:0/2218100763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84103e30 msgr2=0x7f7c8419fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c83fff700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84103e30 0x7f7c8419fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c83fff700 1 -- 172.21.15.102:0/2218100763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c84102290 msgr2=0x7f7c84198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c83fff700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c84102290 0x7f7c84198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c83fff700 1 -- 172.21.15.102:0/2218100763 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c841a00d0 con 0x7f7c84103490 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c83fff700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84103490 0x7f7c841989b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000b560 tx=0x7f7c7000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c81ffb700 1 -- 172.21.15.102:0/2218100763 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70015020 con 0x7f7c84103490 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c81ffb700 1 -- 172.21.15.102:0/2218100763 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c70011ce0 con 0x7f7c84103490 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c841a03c0 con 0x7f7c84103490 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.724+0000 7f7c81ffb700 1 -- 172.21.15.102:0/2218100763 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70010d60 con 0x7f7c84103490 2024-04-24T23:12:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.725+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c841a0980 con 0x7f7c84103490 2024-04-24T23:12:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.726+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c84068f20 con 0x7f7c84103490 2024-04-24T23:12:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.727+0000 7f7c81ffb700 1 -- 172.21.15.102:0/2218100763 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7c70018020 con 0x7f7c84103490 2024-04-24T23:12:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.727+0000 7f7c81ffb700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7c740a2c20 0x7f7c740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.727+0000 7f7c88c1d700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7c740a2c20 0x7f7c740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.727+0000 7f7c81ffb700 1 -- 172.21.15.102:0/2218100763 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7c700ce840 con 0x7f7c84103490 2024-04-24T23:12:42.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.728+0000 7f7c88c1d700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7c740a2c20 0x7f7c740a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7c78000c00 tx=0x7f7c78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.732+0000 7f7c81ffb700 1 -- 172.21.15.102:0/2218100763 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7c7001f020 con 0x7f7c84103490 2024-04-24T23:12:42.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.937+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 --> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f7c84062670 con 0x7f7c740a2c20 2024-04-24T23:12:42.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.939+0000 7f7c81ffb700 1 -- 172.21.15.102:0/2218100763 <== mgr.14152 v2:172.21.15.102:6800/3611055373 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f7c84062670 con 0x7f7c740a2c20 2024-04-24T23:12:42.940 INFO:teuthology.orchestra.run.smithi102.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T23:12:42.940 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T23:12:42.940 INFO:teuthology.orchestra.run.smithi102.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T23:12:42.940 INFO:teuthology.orchestra.run.smithi102.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-24T23:12:42.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7c740a2c20 msgr2=0x7f7c740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7c740a2c20 0x7f7c740a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7c78000c00 tx=0x7f7c78005fb0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84103490 msgr2=0x7f7c841989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84103490 0x7f7c841989b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000b560 tx=0x7f7c7000b870 comp rx=0 tx=0).stop 2024-04-24T23:12:42.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 shutdown_connections 2024-04-24T23:12:42.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84103e30 0x7f7c8419fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c84102290 0x7f7c84198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7c740a2c20 0x7f7c740a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 --2- 172.21.15.102:0/2218100763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c84103490 0x7f7c841989b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 >> 172.21.15.102:0/2218100763 conn(0x7f7c840fd830 msgr2=0x7f7c84106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 shutdown_connections 2024-04-24T23:12:42.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:42.941+0000 7f7c8ae81700 1 -- 172.21.15.102:0/2218100763 wait complete. 2024-04-24T23:12:43.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:43 smithi102 ceph-mon[29216]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-24T23:12:43.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:43 smithi161 ceph-mon[32709]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-24T23:12:43.686 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:12:43.695 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:12:43.695 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T23:12:43.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:43 smithi183 ceph-mon[33137]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-24T23:12:43.933 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:44.308 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:44 smithi102 ceph-mon[29216]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:44.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:44 smithi161 ceph-mon[32709]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:44.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:44 smithi183 ceph-mon[33137]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:45.486 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:45 smithi102 ceph-mon[29216]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:12:45.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:45 smithi161 ceph-mon[32709]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:12:45.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.696+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1926106371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1023b0 msgr2=0x7fcc4c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:45.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.696+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1926106371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1023b0 0x7fcc4c102810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0099c0 tx=0x7fcc3c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.696+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1926106371 shutdown_connections 2024-04-24T23:12:45.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.696+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1926106371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c102d50 0x7fcc4c109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.696+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1926106371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc4c1011b0 0x7fcc4c1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.696+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1926106371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1023b0 0x7fcc4c102810 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.696+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1926106371 >> 172.21.15.102:0/1926106371 conn(0x7fcc4c0fc790 msgr2=0x7fcc4c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:45.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.696+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1926106371 shutdown_connections 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.696+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1926106371 wait complete. 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc50fb7700 1 Processor -- start 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc50fb7700 1 -- start start 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc50fb7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1011b0 0x7fcc4c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc50fb7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c1023b0 0x7fcc4c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc50fb7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc4c102d50 0x7fcc4c19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc50fb7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10ccb0 con 0x7fcc4c1011b0 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc50fb7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10cb30 con 0x7fcc4c1023b0 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc50fb7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10ce30 con 0x7fcc4c102d50 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc4a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1011b0 0x7fcc4c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc4a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1011b0 0x7fcc4c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56376/0 (socket says 172.21.15.102:56376) 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc4a59c700 1 -- 172.21.15.102:0/1958454237 learned_addr learned my addr 172.21.15.102:0/1958454237 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:45.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc4ad9d700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc4c102d50 0x7fcc4c19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc49d9b700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c1023b0 0x7fcc4c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc4a59c700 1 -- 172.21.15.102:0/1958454237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc4c102d50 msgr2=0x7fcc4c19f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc4a59c700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc4c102d50 0x7fcc4c19f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc4a59c700 1 -- 172.21.15.102:0/1958454237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c1023b0 msgr2=0x7fcc4c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc4a59c700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c1023b0 0x7fcc4c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc4a59c700 1 -- 172.21.15.102:0/1958454237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c1a0070 con 0x7fcc4c1011b0 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.697+0000 7fcc49d9b700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c1023b0 0x7fcc4c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.698+0000 7fcc4a59c700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1011b0 0x7fcc4c198460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000c9a0 tx=0x7fcc4000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.698+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1958454237 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40004220 con 0x7fcc4c1011b0 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.698+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c1a0360 con 0x7fcc4c1011b0 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.698+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1958454237 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc40007d10 con 0x7fcc4c1011b0 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.698+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1958454237 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40003e10 con 0x7fcc4c1011b0 2024-04-24T23:12:45.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.698+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c1a0970 con 0x7fcc4c1011b0 2024-04-24T23:12:45.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.699+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c068f20 con 0x7fcc4c1011b0 2024-04-24T23:12:45.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.701+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1958454237 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcc40007860 con 0x7fcc4c1011b0 2024-04-24T23:12:45.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.701+0000 7fcc3b7fe700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc340a2c20 0x7fcc340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:45.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.701+0000 7fcc49d9b700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc340a2c20 0x7fcc340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:45.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.701+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1958454237 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fcc400cf040 con 0x7fcc4c1011b0 2024-04-24T23:12:45.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.702+0000 7fcc49d9b700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc340a2c20 0x7fcc340a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006010 tx=0x7fcc3c005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:45.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.705+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1958454237 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc40014030 con 0x7fcc4c1011b0 2024-04-24T23:12:45.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:45 smithi183 ceph-mon[33137]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:12:45.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.945+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fcc4c1a0c30 con 0x7fcc4c1011b0 2024-04-24T23:12:45.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.945+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1958454237 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3058 (secure 0 0 0) 0x7fcc4009bf90 con 0x7fcc4c1011b0 2024-04-24T23:12:45.948 INFO:teuthology.orchestra.run.smithi102.stdout:e20 2024-04-24T23:12:45.948 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:12:45.948 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:12:45.948 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-04-24T23:12:45.948 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:45.948 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:12:45.948 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-04-24T23:12:45.948 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 20 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-04-24T23:11:00.643113+0000 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-04-24T23:11:04.856565+0000 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 300 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 0 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 3 2024-04-24T23:12:45.949 INFO:teuthology.orchestra.run.smithi102.stdout:in 0,1,2 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=24497,1=24473,2=24472} 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 0 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 24497 members: 24473,24497,24472 2024-04-24T23:12:45.950 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.b{0:24497} state up:active seq 3 addr [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.c{0:24457} state up:standby-replay seq 1 addr [v2:172.21.15.183:6836/3933349751,v1:172.21.15.183:6837/3933349751] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.h{1:24473} state up:active seq 6 addr [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.e{1:24443} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/2778927873,v1:172.21.15.161:6835/2778927873] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.i{2:24472} state up:active seq 6 addr [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.f{2:24442} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/2266462643,v1:172.21.15.183:6835/2266462643] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.102:6834/1487977595,v1:172.21.15.102:6835/1487977595] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.102:6836/1957522131,v1:172.21.15.102:6837/1957522131] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.951 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.g{-1:14622} state up:standby seq 1 addr [v2:172.21.15.102:6838/881198064,v1:172.21.15.102:6839/881198064] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.952 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a{-1:14640} state up:standby seq 1 addr [v2:172.21.15.102:6840/467071210,v1:172.21.15.102:6841/467071210] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.952 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.l{-1:24400} state up:standby seq 1 addr [v2:172.21.15.183:6832/2514459470,v1:172.21.15.183:6833/2514459470] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.952 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.k{-1:24413} state up:standby seq 1 addr [v2:172.21.15.161:6832/756969355,v1:172.21.15.161:6833/756969355] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:12:45.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.947+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc340a2c20 msgr2=0x7fcc340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:45.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.947+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc340a2c20 0x7fcc340a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006010 tx=0x7fcc3c005c40 comp rx=0 tx=0).stop 2024-04-24T23:12:45.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.947+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1011b0 msgr2=0x7fcc4c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:45.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.947+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1011b0 0x7fcc4c198460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000c9a0 tx=0x7fcc4000cd60 comp rx=0 tx=0).stop 2024-04-24T23:12:45.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.948+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 shutdown_connections 2024-04-24T23:12:45.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.948+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc4c102d50 0x7fcc4c19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.948+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc340a2c20 0x7fcc340a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.948+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c1011b0 0x7fcc4c198460 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.948+0000 7fcc50fb7700 1 --2- 172.21.15.102:0/1958454237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c1023b0 0x7fcc4c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:45.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.948+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 >> 172.21.15.102:0/1958454237 conn(0x7fcc4c0fc790 msgr2=0x7fcc4c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:45.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.948+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 shutdown_connections 2024-04-24T23:12:45.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:45.948+0000 7fcc50fb7700 1 -- 172.21.15.102:0/1958454237 wait complete. 2024-04-24T23:12:45.954 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-04-24T23:12:46.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:46 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1958454237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:12:46.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:46 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1958454237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:12:46.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:46 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1958454237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:12:46.773 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-24T23:12:47.013 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:47.457 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:47 smithi102 ceph-mon[29216]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:47 smithi161 ceph-mon[32709]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:47.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:47 smithi183 ceph-mon[33137]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:12:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.867+0000 7fa46c853700 1 -- 172.21.15.102:0/3380029356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa468100340 msgr2=0x7fa4681007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.867+0000 7fa46c853700 1 --2- 172.21.15.102:0/3380029356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa468100340 0x7fa4681007a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009990 tx=0x7fa45c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.867+0000 7fa46c853700 1 -- 172.21.15.102:0/3380029356 shutdown_connections 2024-04-24T23:12:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.867+0000 7fa46c853700 1 --2- 172.21.15.102:0/3380029356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa468100ce0 0x7fa46810a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.867+0000 7fa46c853700 1 --2- 172.21.15.102:0/3380029356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4681038c0 0x7fa468103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.867+0000 7fa46c853700 1 --2- 172.21.15.102:0/3380029356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa468100340 0x7fa4681007a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.867+0000 7fa46c853700 1 -- 172.21.15.102:0/3380029356 >> 172.21.15.102:0/3380029356 conn(0x7fa4680fbea0 msgr2=0x7fa4680fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.867+0000 7fa46c853700 1 -- 172.21.15.102:0/3380029356 shutdown_connections 2024-04-24T23:12:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.867+0000 7fa46c853700 1 -- 172.21.15.102:0/3380029356 wait complete. 2024-04-24T23:12:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa46c853700 1 Processor -- start 2024-04-24T23:12:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa46c853700 1 -- start start 2024-04-24T23:12:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa46c853700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468100340 0x7fa4681982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa46c853700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa468100ce0 0x7fa468198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa46c853700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4681038c0 0x7fa46819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa46c853700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810dd00 con 0x7fa4681038c0 2024-04-24T23:12:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa46c853700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810db80 con 0x7fa468100340 2024-04-24T23:12:48.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa46c853700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa46810de80 con 0x7fa468100ce0 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa466d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4681038c0 0x7fa46819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa466d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4681038c0 0x7fa46819f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53284/0 (socket says 172.21.15.102:53284) 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa466d9d700 1 -- 172.21.15.102:0/504596025 learned_addr learned my addr 172.21.15.102:0/504596025 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa46659c700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468100340 0x7fa4681982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa465d9b700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa468100ce0 0x7fa468198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa466d9d700 1 -- 172.21.15.102:0/504596025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa468100ce0 msgr2=0x7fa468198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa466d9d700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa468100ce0 0x7fa468198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa466d9d700 1 -- 172.21.15.102:0/504596025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468100340 msgr2=0x7fa4681982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa466d9d700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468100340 0x7fa4681982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.868+0000 7fa466d9d700 1 -- 172.21.15.102:0/504596025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46819ff20 con 0x7fa4681038c0 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.869+0000 7fa466d9d700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4681038c0 0x7fa46819f880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa460007da0 tx=0x7fa46000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.869+0000 7fa4577fe700 1 -- 172.21.15.102:0/504596025 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460016020 con 0x7fa4681038c0 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.869+0000 7fa4577fe700 1 -- 172.21.15.102:0/504596025 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa46000ad00 con 0x7fa4681038c0 2024-04-24T23:12:48.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.869+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4681a0210 con 0x7fa4681038c0 2024-04-24T23:12:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.869+0000 7fa4577fe700 1 -- 172.21.15.102:0/504596025 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4600115e0 con 0x7fa4681038c0 2024-04-24T23:12:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.869+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4681a0700 con 0x7fa4681038c0 2024-04-24T23:12:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.870+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa468068f20 con 0x7fa4681038c0 2024-04-24T23:12:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.871+0000 7fa4577fe700 1 -- 172.21.15.102:0/504596025 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa460019020 con 0x7fa4681038c0 2024-04-24T23:12:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.871+0000 7fa4577fe700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4500a2c20 0x7fa4500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.871+0000 7fa46659c700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4500a2c20 0x7fa4500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.872+0000 7fa4577fe700 1 -- 172.21.15.102:0/504596025 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa4600cf410 con 0x7fa4681038c0 2024-04-24T23:12:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.872+0000 7fa46659c700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4500a2c20 0x7fa4500a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa46806a6c0 tx=0x7fa458009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:48.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:48.875+0000 7fa4577fe700 1 -- 172.21.15.102:0/504596025 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa460020020 con 0x7fa4681038c0 2024-04-24T23:12:49.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.073+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa46802db60 con 0x7fa4681038c0 2024-04-24T23:12:49.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.074+0000 7fa4577fe700 1 -- 172.21.15.102:0/504596025 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5340 (secure 0 0 0) 0x7fa46009c360 con 0x7fa4681038c0 2024-04-24T23:12:49.077 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 88 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:fsid 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-04-24T23:02:53.253279+0000 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-04-24T23:11:49.323161+0000 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:crush_version 26 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:full_ratio 0.95 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:backfillfull_ratio 0.9 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:nearfull_ratio 0.85 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:require_min_compat_client luminous 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:min_compat_client luminous 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:require_osd_release squid 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:stretch_mode_enabled false 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-24T23:12:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:max_osd 12 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6802/825890094,v1:172.21.15.102:6803/825890094] [v2:172.21.15.102:6804/825890094,v1:172.21.15.102:6805/825890094] exists,up fabc1206-39d7-4c69-9633-e64bf8677c8c 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/1328568264,v1:172.21.15.161:6801/1328568264] [v2:172.21.15.161:6802/1328568264,v1:172.21.15.161:6803/1328568264] exists,up 0fbfe050-2056-4fe6-91d6-ab6f6e9702b0 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/1173587665,v1:172.21.15.183:6801/1173587665] [v2:172.21.15.183:6802/1173587665,v1:172.21.15.183:6803/1173587665] exists,up 673517e2-faa1-4141-9f1c-bb11f16713d1 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6810/2459908351,v1:172.21.15.102:6811/2459908351] [v2:172.21.15.102:6812/2459908351,v1:172.21.15.102:6813/2459908351] exists,up aea12510-9a47-49d7-b0ca-ee38b19c3249 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/1666138932,v1:172.21.15.161:6809/1666138932] [v2:172.21.15.161:6810/1666138932,v1:172.21.15.161:6811/1666138932] exists,up f1d828f2-40e4-4c0b-9ffe-41ceb2115033 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/351285214,v1:172.21.15.183:6809/351285214] [v2:172.21.15.183:6810/351285214,v1:172.21.15.183:6811/351285214] exists,up f735dd7c-0d04-4448-846f-53949f04a067 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6818/3573356846,v1:172.21.15.102:6819/3573356846] [v2:172.21.15.102:6820/3573356846,v1:172.21.15.102:6821/3573356846] exists,up 74c01493-6457-48d8-807a-0c6108e3d384 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/1449722417,v1:172.21.15.161:6817/1449722417] [v2:172.21.15.161:6818/1449722417,v1:172.21.15.161:6819/1449722417] exists,up 761ed995-553c-4b96-9a90-aec367743007 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/3691469531,v1:172.21.15.183:6817/3691469531] [v2:172.21.15.183:6818/3691469531,v1:172.21.15.183:6819/3691469531] exists,up a6396e17-1338-4c85-8e94-523088a41caf 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6826/3638070029,v1:172.21.15.102:6827/3638070029] [v2:172.21.15.102:6828/3638070029,v1:172.21.15.102:6829/3638070029] exists,up e7df24c5-790f-45d6-a588-4de56ecc06d5 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/968108427,v1:172.21.15.161:6825/968108427] [v2:172.21.15.161:6826/968108427,v1:172.21.15.161:6827/968108427] exists,up 442f5937-b88f-4000-b961-d097cceb9f88 2024-04-24T23:12:49.079 INFO:teuthology.orchestra.run.smithi102.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/2953790548,v1:172.21.15.183:6825/2953790548] [v2:172.21.15.183:6826/2953790548,v1:172.21.15.183:6827/2953790548] exists,up 46d330b0-0ac0-4bba-b07e-f77fb2a202b8 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:pg_upmap_items 2.a [2,11] 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:pg_upmap_items 2.14 [2,4] 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:pg_upmap_items 2.1a [8,4] 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:pg_upmap_items 3.0 [5,9] 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:pg_upmap_items 3.6 [4,6] 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:pg_upmap_items 3.17 [4,8] 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:pg_upmap_items 3.1d [4,7] 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:pg_upmap_items 3.34 [4,9] 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:pg_upmap_items 3.38 [3,9,5,8] 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:0/3958169470 expires 2024-04-25T23:03:48.134719+0000 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:6801/3017038970 expires 2024-04-25T23:03:48.134719+0000 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:0/3702328937 expires 2024-04-25T23:03:21.040554+0000 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:0/1394402685 expires 2024-04-25T23:03:48.134719+0000 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:0/4171285045 expires 2024-04-25T23:03:21.040554+0000 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:6800/3017038970 expires 2024-04-25T23:03:48.134719+0000 2024-04-24T23:12:49.080 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:0/2445922755 expires 2024-04-25T23:03:21.040554+0000 2024-04-24T23:12:49.081 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:0/1700818231 expires 2024-04-25T23:03:48.134719+0000 2024-04-24T23:12:49.081 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:6800/2363268578 expires 2024-04-25T23:03:21.040554+0000 2024-04-24T23:12:49.081 INFO:teuthology.orchestra.run.smithi102.stdout:blocklist 172.21.15.102:6801/2363268578 expires 2024-04-25T23:03:21.040554+0000 2024-04-24T23:12:49.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4500a2c20 msgr2=0x7fa4500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4500a2c20 0x7fa4500a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa46806a6c0 tx=0x7fa458009260 comp rx=0 tx=0).stop 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4681038c0 msgr2=0x7fa46819f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4681038c0 0x7fa46819f880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa460007da0 tx=0x7fa46000a340 comp rx=0 tx=0).stop 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 shutdown_connections 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa4500a2c20 0x7fa4500a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4681038c0 0x7fa46819f880 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468100340 0x7fa4681982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 --2- 172.21.15.102:0/504596025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa468100ce0 0x7fa468198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.077+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 >> 172.21.15.102:0/504596025 conn(0x7fa4680fbea0 msgr2=0x7fa468106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.078+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 shutdown_connections 2024-04-24T23:12:49.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.078+0000 7fa46c853700 1 -- 172.21.15.102:0/504596025 wait complete. 2024-04-24T23:12:49.517 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:49 smithi102 ceph-mon[29216]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:49.517 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:49 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-24T23:12:49.517 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:49 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/504596025' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:12:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:49 smithi161 ceph-mon[32709]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:49 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-24T23:12:49.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:49 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/504596025' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:12:49.741 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-24T23:12:49.753 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:49 smithi183 ceph-mon[33137]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:12:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:49 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-24T23:12:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:49 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/504596025' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:12:49.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 -- 172.21.15.102:0/662146504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 msgr2=0x7f8250108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 --2- 172.21.15.102:0/662146504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 0x7f8250108ae0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8240009900 tx=0x7f824002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:49.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 -- 172.21.15.102:0/662146504 shutdown_connections 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 --2- 172.21.15.102:0/662146504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 0x7f8250108ae0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 --2- 172.21.15.102:0/662146504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8250101890 0x7f8250101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 --2- 172.21.15.102:0/662146504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82501006a0 0x7f8250100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 -- 172.21.15.102:0/662146504 >> 172.21.15.102:0/662146504 conn(0x7f82500fbc30 msgr2=0x7f82500fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 -- 172.21.15.102:0/662146504 shutdown_connections 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 -- 172.21.15.102:0/662146504 wait complete. 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.851+0000 7f825586b640 1 Processor -- start 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f825586b640 1 -- start start 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f825586b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82501006a0 0x7f82501045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f825586b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8250101890 0x7f8250102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f825586b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 0x7f82501031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f825586b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f825010c240 con 0x7f8250102250 2024-04-24T23:12:49.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f825586b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f825010c0c0 con 0x7f8250101890 2024-04-24T23:12:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f825586b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f825010c3c0 con 0x7f82501006a0 2024-04-24T23:12:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 0x7f82501031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 0x7f82501031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53304/0 (socket says 172.21.15.102:53304) 2024-04-24T23:12:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824f7fe640 1 -- 172.21.15.102:0/4116211504 learned_addr learned my addr 172.21.15.102:0/4116211504 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824e7fc640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8250101890 0x7f8250102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824effd640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82501006a0 0x7f82501045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:49.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824f7fe640 1 -- 172.21.15.102:0/4116211504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82501006a0 msgr2=0x7f82501045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824f7fe640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82501006a0 0x7f82501045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824f7fe640 1 -- 172.21.15.102:0/4116211504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8250101890 msgr2=0x7f8250102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824f7fe640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8250101890 0x7f8250102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824f7fe640 1 -- 172.21.15.102:0/4116211504 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82501039d0 con 0x7f8250102250 2024-04-24T23:12:49.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824e7fc640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8250101890 0x7f8250102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:49.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f824f7fe640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 0x7f82501031b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8240005940 tx=0x7f82400043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:49.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f8254869640 1 -- 172.21.15.102:0/4116211504 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824003f030 con 0x7f8250102250 2024-04-24T23:12:49.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f8254869640 1 -- 172.21.15.102:0/4116211504 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8240003cb0 con 0x7f8250102250 2024-04-24T23:12:49.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82501a1aa0 con 0x7f8250102250 2024-04-24T23:12:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f8254869640 1 -- 172.21.15.102:0/4116211504 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240035070 con 0x7f8250102250 2024-04-24T23:12:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.852+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82501a1f90 con 0x7f8250102250 2024-04-24T23:12:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.854+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82501a2220 con 0x7f8250102250 2024-04-24T23:12:49.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.855+0000 7f8254869640 1 -- 172.21.15.102:0/4116211504 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8240042030 con 0x7f8250102250 2024-04-24T23:12:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.855+0000 7f8254869640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82380a2b90 0x7f82380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.855+0000 7f824effd640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82380a2b90 0x7f82380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.856+0000 7f8254869640 1 -- 172.21.15.102:0/4116211504 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f82400ef820 con 0x7f8250102250 2024-04-24T23:12:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.856+0000 7f824effd640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82380a2b90 0x7f82380a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f82501016f0 tx=0x7f82480094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:49.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:49.860+0000 7f8254869640 1 -- 172.21.15.102:0/4116211504 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82400bc750 con 0x7f8250102250 2024-04-24T23:12:50.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.040+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82501a2990 con 0x7f8250102250 2024-04-24T23:12:50.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.041+0000 7f8254869640 1 -- 172.21.15.102:0/4116211504 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22767 (secure 0 0 0) 0x7f82400bc050 con 0x7f8250102250 2024-04-24T23:12:50.044 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:50.044 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":89,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:12:49.311433+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:50.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82380a2b90 msgr2=0x7f82380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82380a2b90 0x7f82380a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f82501016f0 tx=0x7f82480094f0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 msgr2=0x7f82501031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 0x7f82501031b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8240005940 tx=0x7f82400043c0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 shutdown_connections 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82380a2b90 0x7f82380a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8250102250 0x7f82501031b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8250101890 0x7f8250102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 --2- 172.21.15.102:0/4116211504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82501006a0 0x7f82501045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 >> 172.21.15.102:0/4116211504 conn(0x7f82500fbc30 msgr2=0x7f82500fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 shutdown_connections 2024-04-24T23:12:50.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.043+0000 7f825586b640 1 -- 172.21.15.102:0/4116211504 wait complete. 2024-04-24T23:12:50.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:58.730586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:59.855279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:12:50.058 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:50.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 -- 172.21.15.102:0/858985752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 msgr2=0x7f7dbc101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/858985752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 0x7f7dbc101bb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009900 tx=0x7f7dac02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 -- 172.21.15.102:0/858985752 shutdown_connections 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/858985752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dbc103340 0x7f7dbc109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/858985752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dbc102980 0x7f7dbc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/858985752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 0x7f7dbc101bb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 -- 172.21.15.102:0/858985752 >> 172.21.15.102:0/858985752 conn(0x7f7dbc0fcd20 msgr2=0x7f7dbc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 -- 172.21.15.102:0/858985752 shutdown_connections 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 -- 172.21.15.102:0/858985752 wait complete. 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 Processor -- start 2024-04-24T23:12:50.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.157+0000 7f7dc21c1640 1 -- start start 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dc21c1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 0x7f7dbc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dc21c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dbc102980 0x7f7dbc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dc21c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dbc103340 0x7f7dbc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dc21c1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc10d2c0 con 0x7f7dbc101790 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dc21c1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc10d140 con 0x7f7dbc102980 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dc21c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc10d440 con 0x7f7dbc103340 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 0x7f7dbc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 0x7f7dbc197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53314/0 (socket says 172.21.15.102:53314) 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbb7fe640 1 -- 172.21.15.102:0/692409718 learned_addr learned my addr 172.21.15.102:0/692409718 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbbfff640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dbc103340 0x7f7dbc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7db3fff640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dbc102980 0x7f7dbc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbb7fe640 1 -- 172.21.15.102:0/692409718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dbc103340 msgr2=0x7f7dbc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbb7fe640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dbc103340 0x7f7dbc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbb7fe640 1 -- 172.21.15.102:0/692409718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dbc102980 msgr2=0x7f7dbc197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbb7fe640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dbc102980 0x7f7dbc197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbb7fe640 1 -- 172.21.15.102:0/692409718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc19f660 con 0x7f7dbc101790 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dbb7fe640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 0x7f7dbc197a40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009230 tx=0x7f7dac004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7db97fa640 1 -- 172.21.15.102:0/692409718 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac03f030 con 0x7f7dbc101790 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7db97fa640 1 -- 172.21.15.102:0/692409718 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dac003cb0 con 0x7f7dbc101790 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc19f8f0 con 0x7f7dbc101790 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7db97fa640 1 -- 172.21.15.102:0/692409718 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac035070 con 0x7f7dbc101790 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.158+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc19fde0 con 0x7f7dbc101790 2024-04-24T23:12:50.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.159+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc108c40 con 0x7f7dbc101790 2024-04-24T23:12:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.160+0000 7f7db97fa640 1 -- 172.21.15.102:0/692409718 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7dac042030 con 0x7f7dbc101790 2024-04-24T23:12:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.161+0000 7f7db97fa640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7da80a2b90 0x7f7da80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.161+0000 7f7db3fff640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7da80a2b90 0x7f7da80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.161+0000 7f7db97fa640 1 -- 172.21.15.102:0/692409718 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f7dac0efb80 con 0x7f7dbc101790 2024-04-24T23:12:50.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.161+0000 7f7db3fff640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7da80a2b90 0x7f7da80a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc198f50 tx=0x7f7da4005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.165+0000 7f7db97fa640 1 -- 172.21.15.102:0/692409718 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7dac0bcab0 con 0x7f7dbc101790 2024-04-24T23:12:50.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.345+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7dbc062340 con 0x7f7dbc101790 2024-04-24T23:12:50.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.346+0000 7f7db97fa640 1 -- 172.21.15.102:0/692409718 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f7dac0bc3b0 con 0x7f7dbc101790 2024-04-24T23:12:50.348 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:12:50.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7da80a2b90 msgr2=0x7f7da80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7da80a2b90 0x7f7da80a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc198f50 tx=0x7f7da4005c60 comp rx=0 tx=0).stop 2024-04-24T23:12:50.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 msgr2=0x7f7dbc197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 0x7f7dbc197a40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009230 tx=0x7f7dac004340 comp rx=0 tx=0).stop 2024-04-24T23:12:50.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 shutdown_connections 2024-04-24T23:12:50.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7da80a2b90 0x7f7da80a5050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dbc103340 0x7f7dbc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dbc102980 0x7f7dbc197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 --2- 172.21.15.102:0/692409718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dbc101790 0x7f7dbc197a40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 >> 172.21.15.102:0/692409718 conn(0x7f7dbc0fcd20 msgr2=0x7f7dbc105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 shutdown_connections 2024-04-24T23:12:50.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.348+0000 7f7dc21c1640 1 -- 172.21.15.102:0/692409718 wait complete. 2024-04-24T23:12:50.361 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:50.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 -- 172.21.15.102:0/230232548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 msgr2=0x7ff48c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 --2- 172.21.15.102:0/230232548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 0x7ff48c100ac0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009900 tx=0x7ff47c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:50.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 -- 172.21.15.102:0/230232548 shutdown_connections 2024-04-24T23:12:50.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 --2- 172.21.15.102:0/230232548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48c102250 0x7ff48c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 --2- 172.21.15.102:0/230232548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff48c101890 0x7ff48c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 --2- 172.21.15.102:0/230232548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 0x7ff48c100ac0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 -- 172.21.15.102:0/230232548 >> 172.21.15.102:0/230232548 conn(0x7ff48c0fbc30 msgr2=0x7ff48c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 -- 172.21.15.102:0/230232548 shutdown_connections 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 -- 172.21.15.102:0/230232548 wait complete. 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 Processor -- start 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.460+0000 7ff4921c4640 1 -- start start 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4921c4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 0x7ff48c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4921c4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48c101890 0x7ff48c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4921c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff48c102250 0x7ff48c19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4921c4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10c2a0 con 0x7ff48c1006a0 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4921c4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10c120 con 0x7ff48c101890 2024-04-24T23:12:50.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4921c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10c420 con 0x7ff48c102250 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff48b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 0x7ff48c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff48b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 0x7ff48c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53320/0 (socket says 172.21.15.102:53320) 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff48b7fe640 1 -- 172.21.15.102:0/2581530111 learned_addr learned my addr 172.21.15.102:0/2581530111 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff483fff640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48c101890 0x7ff48c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff48b7fe640 1 -- 172.21.15.102:0/2581530111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff48c102250 msgr2=0x7ff48c19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff48b7fe640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff48c102250 0x7ff48c19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff48b7fe640 1 -- 172.21.15.102:0/2581530111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48c101890 msgr2=0x7ff48c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff48b7fe640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48c101890 0x7ff48c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff48b7fe640 1 -- 172.21.15.102:0/2581530111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c19f7d0 con 0x7ff48c1006a0 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff483fff640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48c101890 0x7ff48c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff48b7fe640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 0x7ff48c197c50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009230 tx=0x7ff47c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4897fa640 1 -- 172.21.15.102:0/2581530111 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c03f030 con 0x7ff48c1006a0 2024-04-24T23:12:50.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4897fa640 1 -- 172.21.15.102:0/2581530111 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff47c003cb0 con 0x7ff48c1006a0 2024-04-24T23:12:50.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c19fa60 con 0x7ff48c1006a0 2024-04-24T23:12:50.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4897fa640 1 -- 172.21.15.102:0/2581530111 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c035070 con 0x7ff48c1006a0 2024-04-24T23:12:50.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.461+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c19ffd0 con 0x7ff48c1006a0 2024-04-24T23:12:50.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.462+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48c1a01a0 con 0x7ff48c1006a0 2024-04-24T23:12:50.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.463+0000 7ff4897fa640 1 -- 172.21.15.102:0/2581530111 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff47c042030 con 0x7ff48c1006a0 2024-04-24T23:12:50.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.464+0000 7ff4897fa640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff4780a2b90 0x7ff4780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.464+0000 7ff4897fa640 1 -- 172.21.15.102:0/2581530111 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff47c0ef7a0 con 0x7ff48c1006a0 2024-04-24T23:12:50.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.464+0000 7ff483fff640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff4780a2b90 0x7ff4780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.464+0000 7ff483fff640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff4780a2b90 0x7ff4780a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff474006fd0 tx=0x7ff474008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.468+0000 7ff4897fa640 1 -- 172.21.15.102:0/2581530111 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff47c0bc6d0 con 0x7ff48c1006a0 2024-04-24T23:12:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:50 smithi161 ceph-mon[32709]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-24T23:12:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:50 smithi161 ceph-mon[32709]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:12:50.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:50 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4116211504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:50.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.646+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff48c1a0370 con 0x7ff48c1006a0 2024-04-24T23:12:50.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.646+0000 7ff4897fa640 1 -- 172.21.15.102:0/2581530111 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7ff47c0bbfd0 con 0x7ff48c1006a0 2024-04-24T23:12:50.648 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.648+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff4780a2b90 msgr2=0x7ff4780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.648+0000 7ff4921c4640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff4780a2b90 0x7ff4780a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff474006fd0 tx=0x7ff474008040 comp rx=0 tx=0).stop 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 msgr2=0x7ff48c197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 0x7ff48c197c50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009230 tx=0x7ff47c004340 comp rx=0 tx=0).stop 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 shutdown_connections 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7ff4780a2b90 0x7ff4780a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff48c102250 0x7ff48c19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48c101890 0x7ff48c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 --2- 172.21.15.102:0/2581530111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff48c1006a0 0x7ff48c197c50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 >> 172.21.15.102:0/2581530111 conn(0x7ff48c0fbc30 msgr2=0x7ff48c069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 shutdown_connections 2024-04-24T23:12:50.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.649+0000 7ff4921c4640 1 -- 172.21.15.102:0/2581530111 wait complete. 2024-04-24T23:12:50.661 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:50 smithi102 ceph-mon[29216]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-24T23:12:50.661 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:50 smithi102 ceph-mon[29216]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:12:50.661 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:50 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4116211504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:50.662 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:50.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:50 smithi183 ceph-mon[33137]: from='mgr.14152 172.21.15.102:0/1881543838' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-24T23:12:50.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:50 smithi183 ceph-mon[33137]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:12:50.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:50 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4116211504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 -- 172.21.15.102:0/2065017838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80141033e0 msgr2=0x7f8014109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 --2- 172.21.15.102:0/2065017838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80141033e0 0x7f8014109c70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f800c009cd0 tx=0x7f800c02a8e0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 -- 172.21.15.102:0/2065017838 shutdown_connections 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 --2- 172.21.15.102:0/2065017838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80141033e0 0x7f8014109c70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 --2- 172.21.15.102:0/2065017838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8014102a20 0x7f8014102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 --2- 172.21.15.102:0/2065017838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80140ffa50 0x7f80140ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 -- 172.21.15.102:0/2065017838 >> 172.21.15.102:0/2065017838 conn(0x7f80140fb380 msgr2=0x7f80140fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 -- 172.21.15.102:0/2065017838 shutdown_connections 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 -- 172.21.15.102:0/2065017838 wait complete. 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 Processor -- start 2024-04-24T23:12:50.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.761+0000 7f801a314640 1 -- start start 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f801a314640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80140ffa50 0x7f8014197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f801a314640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8014102a20 0x7f8014197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f801a314640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80141033e0 0x7f801419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f801a314640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f801410d2d0 con 0x7f80140ffa50 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f801a314640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f801410d150 con 0x7f80141033e0 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f801a314640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f801410d450 con 0x7f8014102a20 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f8013fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80140ffa50 0x7f8014197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f8013fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80140ffa50 0x7f8014197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53338/0 (socket says 172.21.15.102:53338) 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f8013fff640 1 -- 172.21.15.102:0/567854180 learned_addr learned my addr 172.21.15.102:0/567854180 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f8013fff640 1 -- 172.21.15.102:0/567854180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8014102a20 msgr2=0x7f8014197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f8013fff640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8014102a20 0x7f8014197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f8013fff640 1 -- 172.21.15.102:0/567854180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80141033e0 msgr2=0x7f801419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f8013fff640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80141033e0 0x7f801419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f8013fff640 1 -- 172.21.15.102:0/567854180 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801419f5f0 con 0x7f80140ffa50 2024-04-24T23:12:50.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f8013fff640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80140ffa50 0x7f8014197a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f800400d950 tx=0x7f800400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f80117fa640 1 -- 172.21.15.102:0/567854180 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004017070 con 0x7f80140ffa50 2024-04-24T23:12:50.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f80117fa640 1 -- 172.21.15.102:0/567854180 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8004009c30 con 0x7f80140ffa50 2024-04-24T23:12:50.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801419f8e0 con 0x7f80140ffa50 2024-04-24T23:12:50.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f80117fa640 1 -- 172.21.15.102:0/567854180 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800400f620 con 0x7f80140ffa50 2024-04-24T23:12:50.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.762+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801419fde0 con 0x7f80140ffa50 2024-04-24T23:12:50.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.764+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8014108c70 con 0x7f80140ffa50 2024-04-24T23:12:50.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.765+0000 7f80117fa640 1 -- 172.21.15.102:0/567854180 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f800401a070 con 0x7f80140ffa50 2024-04-24T23:12:50.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.765+0000 7f80117fa640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ffc0a2b90 0x7f7ffc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.765+0000 7f80137fe640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ffc0a2b90 0x7f7ffc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.765+0000 7f80117fa640 1 -- 172.21.15.102:0/567854180 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f80040ceb80 con 0x7f80140ffa50 2024-04-24T23:12:50.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.766+0000 7f80137fe640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ffc0a2b90 0x7f7ffc0a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8014198ee0 tx=0x7f8008005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.769+0000 7f80117fa640 1 -- 172.21.15.102:0/567854180 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8004009de0 con 0x7f80140ffa50 2024-04-24T23:12:50.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.945+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8014062440 con 0x7f80140ffa50 2024-04-24T23:12:50.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.945+0000 7f80117fa640 1 -- 172.21.15.102:0/567854180 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f800409bab0 con 0x7f80140ffa50 2024-04-24T23:12:50.948 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-24T23:12:50.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ffc0a2b90 msgr2=0x7f7ffc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ffc0a2b90 0x7f7ffc0a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8014198ee0 tx=0x7f8008005bf0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80140ffa50 msgr2=0x7f8014197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80140ffa50 0x7f8014197a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f800400d950 tx=0x7f800400de10 comp rx=0 tx=0).stop 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 shutdown_connections 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f7ffc0a2b90 0x7f7ffc0a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80141033e0 0x7f801419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8014102a20 0x7f8014197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 --2- 172.21.15.102:0/567854180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80140ffa50 0x7f8014197a20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.947+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 >> 172.21.15.102:0/567854180 conn(0x7f80140fb380 msgr2=0x7f8014105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.948+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 shutdown_connections 2024-04-24T23:12:50.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:50.948+0000 7f801a314640 1 -- 172.21.15.102:0/567854180 wait complete. 2024-04-24T23:12:50.961 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:51.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.056+0000 7f64e6de7640 1 -- 172.21.15.102:0/998392819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 msgr2=0x7f64e0101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.056+0000 7f64e6de7640 1 --2- 172.21.15.102:0/998392819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 0x7f64e0101bb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009900 tx=0x7f64d402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.056+0000 7f64e6de7640 1 -- 172.21.15.102:0/998392819 shutdown_connections 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.056+0000 7f64e6de7640 1 --2- 172.21.15.102:0/998392819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e0103340 0x7f64e0109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.056+0000 7f64e6de7640 1 --2- 172.21.15.102:0/998392819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e0102980 0x7f64e0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.056+0000 7f64e6de7640 1 --2- 172.21.15.102:0/998392819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 0x7f64e0101bb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.056+0000 7f64e6de7640 1 -- 172.21.15.102:0/998392819 >> 172.21.15.102:0/998392819 conn(0x7f64e00fcd20 msgr2=0x7f64e00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.056+0000 7f64e6de7640 1 -- 172.21.15.102:0/998392819 shutdown_connections 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.056+0000 7f64e6de7640 1 -- 172.21.15.102:0/998392819 wait complete. 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e6de7640 1 Processor -- start 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e6de7640 1 -- start start 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e6de7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 0x7f64e01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e6de7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e0102980 0x7f64e0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e6de7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e0103340 0x7f64e019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e6de7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64e010d290 con 0x7f64e0101790 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e6de7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f64e010d110 con 0x7f64e0103340 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e6de7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64e010d410 con 0x7f64e0102980 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e4b83640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 0x7f64e01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e4b83640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 0x7f64e01979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53356/0 (socket says 172.21.15.102:53356) 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e4b83640 1 -- 172.21.15.102:0/786073401 learned_addr learned my addr 172.21.15.102:0/786073401 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e5384640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e0103340 0x7f64e019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e4b83640 1 -- 172.21.15.102:0/786073401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e0102980 msgr2=0x7f64e0197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e4b83640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e0102980 0x7f64e0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e4b83640 1 -- 172.21.15.102:0/786073401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e0103340 msgr2=0x7f64e019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e4b83640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e0103340 0x7f64e019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e4b83640 1 -- 172.21.15.102:0/786073401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e019f5e0 con 0x7f64e0101790 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e4b83640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 0x7f64e01979c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f64d40314e0 tx=0x7f64d40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64ddffb640 1 -- 172.21.15.102:0/786073401 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d403f030 con 0x7f64e0101790 2024-04-24T23:12:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64ddffb640 1 -- 172.21.15.102:0/786073401 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64d4003cb0 con 0x7f64e0101790 2024-04-24T23:12:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64ddffb640 1 -- 172.21.15.102:0/786073401 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4035070 con 0x7f64e0101790 2024-04-24T23:12:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.057+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e019f870 con 0x7f64e0101790 2024-04-24T23:12:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.058+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e019fdb0 con 0x7f64e0101790 2024-04-24T23:12:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.059+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e0108b80 con 0x7f64e0101790 2024-04-24T23:12:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.060+0000 7f64ddffb640 1 -- 172.21.15.102:0/786073401 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f64d4042030 con 0x7f64e0101790 2024-04-24T23:12:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.060+0000 7f64ddffb640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64d00a2c10 0x7f64d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.060+0000 7f64dffff640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64d00a2c10 0x7f64d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.061+0000 7f64ddffb640 1 -- 172.21.15.102:0/786073401 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f64d40ef6b0 con 0x7f64e0101790 2024-04-24T23:12:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.061+0000 7f64dffff640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64d00a2c10 0x7f64d00a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f64e0198ed0 tx=0x7f64cc005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:51.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.065+0000 7f64ddffb640 1 -- 172.21.15.102:0/786073401 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f64d40bc5e0 con 0x7f64e0101790 2024-04-24T23:12:51.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.278+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64e002d170 con 0x7f64e0101790 2024-04-24T23:12:51.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.279+0000 7f64ddffb640 1 -- 172.21.15.102:0/786073401 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7f64d40bbee0 con 0x7f64e0101790 2024-04-24T23:12:51.281 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:51.281 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:11:00.643113+0000","modified":"2024-04-24T23:11:04.856565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/1258581930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1258581930},{"type":"v1","addr":"172.21.15.161:6839","nonce":1258581930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24473,24497,24472]},"id":1}]} 2024-04-24T23:12:51.281 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-04-24T23:12:51.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64d00a2c10 msgr2=0x7f64d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64d00a2c10 0x7f64d00a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f64e0198ed0 tx=0x7f64cc005ed0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 msgr2=0x7f64e01979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 0x7f64e01979c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f64d40314e0 tx=0x7f64d40042e0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 shutdown_connections 2024-04-24T23:12:51.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f64d00a2c10 0x7f64d00a50d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e0103340 0x7f64e019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64e0102980 0x7f64e0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 --2- 172.21.15.102:0/786073401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e0101790 0x7f64e01979c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 >> 172.21.15.102:0/786073401 conn(0x7f64e00fcd20 msgr2=0x7f64e0105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:51.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 shutdown_connections 2024-04-24T23:12:51.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.281+0000 7f64e6de7640 1 -- 172.21.15.102:0/786073401 wait complete. 2024-04-24T23:12:51.295 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:51.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/1263289857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28102250 msgr2=0x7fcc28108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/1263289857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28102250 0x7fcc28108ae0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009ca0 tx=0x7fcc2002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/1263289857 shutdown_connections 2024-04-24T23:12:51.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/1263289857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28102250 0x7fcc28108ae0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/1263289857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc28101890 0x7fcc28101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/1263289857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc281006a0 0x7fcc28100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/1263289857 >> 172.21.15.102:0/1263289857 conn(0x7fcc280fbc30 msgr2=0x7fcc280fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/1263289857 shutdown_connections 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/1263289857 wait complete. 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 Processor -- start 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 -- start start 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc281006a0 0x7fcc28197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28101890 0x7fcc28197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc28102250 0x7fcc2819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2810c1b0 con 0x7fcc28101890 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2810c030 con 0x7fcc28102250 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc2ddf9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2810c330 con 0x7fcc281006a0 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.391+0000 7fcc1ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28101890 0x7fcc28197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc1ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28101890 0x7fcc28197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53378/0 (socket says 172.21.15.102:53378) 2024-04-24T23:12:51.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc1ffff640 1 -- 172.21.15.102:0/4011819924 learned_addr learned my addr 172.21.15.102:0/4011819924 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc277fe640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc281006a0 0x7fcc28197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc27fff640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc28102250 0x7fcc2819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc1ffff640 1 -- 172.21.15.102:0/4011819924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc281006a0 msgr2=0x7fcc28197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc1ffff640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc281006a0 0x7fcc28197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc1ffff640 1 -- 172.21.15.102:0/4011819924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc28102250 msgr2=0x7fcc2819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc1ffff640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc28102250 0x7fcc2819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc1ffff640 1 -- 172.21.15.102:0/4011819924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2819f450 con 0x7fcc28101890 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc27fff640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc28102250 0x7fcc2819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc1ffff640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28101890 0x7fcc28197d70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000c920 tx=0x7fcc1000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc257fa640 1 -- 172.21.15.102:0/4011819924 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10007c70 con 0x7fcc28101890 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc257fa640 1 -- 172.21.15.102:0/4011819924 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc10007e10 con 0x7fcc28101890 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2819f740 con 0x7fcc28101890 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc257fa640 1 -- 172.21.15.102:0/4011819924 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10018820 con 0x7fcc28101890 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.392+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2819fce0 con 0x7fcc28101890 2024-04-24T23:12:51.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.393+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc2819ffa0 con 0x7fcc28101890 2024-04-24T23:12:51.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.394+0000 7fcc257fa640 1 -- 172.21.15.102:0/4011819924 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcc100189c0 con 0x7fcc28101890 2024-04-24T23:12:51.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.395+0000 7fcc257fa640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc140a2c10 0x7fcc140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.395+0000 7fcc277fe640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc140a2c10 0x7fcc140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.395+0000 7fcc257fa640 1 -- 172.21.15.102:0/4011819924 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fcc100cf210 con 0x7fcc28101890 2024-04-24T23:12:51.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.395+0000 7fcc277fe640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc140a2c10 0x7fcc140a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcc180058c0 tx=0x7fcc18005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:51.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.399+0000 7fcc257fa640 1 -- 172.21.15.102:0/4011819924 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc10014030 con 0x7fcc28101890 2024-04-24T23:12:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:51 smithi102 ceph-mon[29216]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:12:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:51 smithi102 ceph-mon[29216]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:12:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:51 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/692409718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:51 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2581530111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:51 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/567854180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:51 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/786073401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:51.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.576+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc281a0710 con 0x7fcc28101890 2024-04-24T23:12:51.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.578+0000 7fcc257fa640 1 -- 172.21.15.102:0/4011819924 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22767 (secure 0 0 0) 0x7fcc1009c140 con 0x7fcc28101890 2024-04-24T23:12:51.580 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:51.580 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":90,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:12:50.317275+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc140a2c10 msgr2=0x7fcc140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc140a2c10 0x7fcc140a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcc180058c0 tx=0x7fcc18005850 comp rx=0 tx=0).stop 2024-04-24T23:12:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28101890 msgr2=0x7fcc28197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28101890 0x7fcc28197d70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000c920 tx=0x7fcc1000cde0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 shutdown_connections 2024-04-24T23:12:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fcc140a2c10 0x7fcc140a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc28102250 0x7fcc2819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc28101890 0x7fcc28197d70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 --2- 172.21.15.102:0/4011819924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc281006a0 0x7fcc28197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 >> 172.21.15.102:0/4011819924 conn(0x7fcc280fbc30 msgr2=0x7fcc280fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:51.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 shutdown_connections 2024-04-24T23:12:51.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.580+0000 7fcc2ddf9640 1 -- 172.21.15.102:0/4011819924 wait complete. 2024-04-24T23:12:51.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:58.730586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:59.855279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:12:51.595 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:51 smithi161 ceph-mon[32709]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:12:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:51 smithi161 ceph-mon[32709]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:12:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:51 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/692409718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:51 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2581530111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:51 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/567854180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:51.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:51 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/786073401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:51.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.693+0000 7f072bcd4640 1 -- 172.21.15.102:0/3441062022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 msgr2=0x7f0724108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.693+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3441062022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 0x7f0724108ac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f071c009ca0 tx=0x7f071c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.693+0000 7f072bcd4640 1 -- 172.21.15.102:0/3441062022 shutdown_connections 2024-04-24T23:12:51.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.693+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3441062022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 0x7f0724108ac0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.693+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3441062022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0724101870 0x7f0724101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.693+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3441062022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0724100680 0x7f0724100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.693+0000 7f072bcd4640 1 -- 172.21.15.102:0/3441062022 >> 172.21.15.102:0/3441062022 conn(0x7f07240fbc30 msgr2=0x7f07240fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:51.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.693+0000 7f072bcd4640 1 -- 172.21.15.102:0/3441062022 shutdown_connections 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.693+0000 7f072bcd4640 1 -- 172.21.15.102:0/3441062022 wait complete. 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072bcd4640 1 Processor -- start 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072bcd4640 1 -- start start 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072bcd4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0724100680 0x7f0724199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072bcd4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0724101870 0x7f072419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072bcd4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 0x7f07241a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072bcd4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f072410c210 con 0x7f0724102230 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072bcd4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f072410c090 con 0x7f0724101870 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072bcd4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f072410c390 con 0x7f0724100680 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072a271640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 0x7f07241a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072a271640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 0x7f07241a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53394/0 (socket says 172.21.15.102:53394) 2024-04-24T23:12:51.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072a271640 1 -- 172.21.15.102:0/3204738571 learned_addr learned my addr 172.21.15.102:0/3204738571 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072926f640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0724101870 0x7f072419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072a271640 1 -- 172.21.15.102:0/3204738571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0724100680 msgr2=0x7f0724199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072a271640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0724100680 0x7f0724199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072a271640 1 -- 172.21.15.102:0/3204738571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0724101870 msgr2=0x7f072419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072a271640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0724101870 0x7f072419a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.694+0000 7f072a271640 1 -- 172.21.15.102:0/3204738571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07241a17a0 con 0x7f0724102230 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.695+0000 7f072a271640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 0x7f07241a1130 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f071c009dc0 tx=0x7f071c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.695+0000 7f071affd640 1 -- 172.21.15.102:0/3204738571 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c004040 con 0x7f0724102230 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.695+0000 7f071affd640 1 -- 172.21.15.102:0/3204738571 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f071c004690 con 0x7f0724102230 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.695+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07241a1a30 con 0x7f0724102230 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.695+0000 7f071affd640 1 -- 172.21.15.102:0/3204738571 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c02f4c0 con 0x7f0724102230 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.695+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07241a1fd0 con 0x7f0724102230 2024-04-24T23:12:51.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.696+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07241a2200 con 0x7f0724102230 2024-04-24T23:12:51.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.697+0000 7f071affd640 1 -- 172.21.15.102:0/3204738571 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f071c030650 con 0x7f0724102230 2024-04-24T23:12:51.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.697+0000 7f071affd640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07100a2c10 0x7f07100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.697+0000 7f0729a70640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07100a2c10 0x7f07100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.698+0000 7f071affd640 1 -- 172.21.15.102:0/3204738571 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f071c0f0a50 con 0x7f0724102230 2024-04-24T23:12:51.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.698+0000 7f0729a70640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07100a2c10 0x7f07100a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f07241016d0 tx=0x7f0720005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:51.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.701+0000 7f071affd640 1 -- 172.21.15.102:0/3204738571 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f071c0bd980 con 0x7f0724102230 2024-04-24T23:12:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:51 smithi183 ceph-mon[33137]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:12:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:51 smithi183 ceph-mon[33137]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:12:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:51 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/692409718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:51 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2581530111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:51 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/567854180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:51.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:51 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/786073401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:51.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.878+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07241a23d0 con 0x7f0724102230 2024-04-24T23:12:51.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.879+0000 7f071affd640 1 -- 172.21.15.102:0/3204738571 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f071c0bd280 con 0x7f0724102230 2024-04-24T23:12:51.881 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07100a2c10 msgr2=0x7f07100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07100a2c10 0x7f07100a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f07241016d0 tx=0x7f0720005c90 comp rx=0 tx=0).stop 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 msgr2=0x7f07241a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 0x7f07241a1130 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f071c009dc0 tx=0x7f071c003ba0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 shutdown_connections 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f07100a2c10 0x7f07100a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102230 0x7f07241a1130 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0724101870 0x7f072419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 --2- 172.21.15.102:0/3204738571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0724100680 0x7f0724199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 >> 172.21.15.102:0/3204738571 conn(0x7f07240fbc30 msgr2=0x7f07240fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:51.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 shutdown_connections 2024-04-24T23:12:51.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.881+0000 7f072bcd4640 1 -- 172.21.15.102:0/3204738571 wait complete. 2024-04-24T23:12:51.894 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.993+0000 7f0cefc15640 1 -- 172.21.15.102:0/1564886852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8102230 msgr2=0x7f0ce8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.993+0000 7f0cefc15640 1 --2- 172.21.15.102:0/1564886852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8102230 0x7f0ce8108ac0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009900 tx=0x7f0ce002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.993+0000 7f0cefc15640 1 -- 172.21.15.102:0/1564886852 shutdown_connections 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.993+0000 7f0cefc15640 1 --2- 172.21.15.102:0/1564886852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8102230 0x7f0ce8108ac0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.993+0000 7f0cefc15640 1 --2- 172.21.15.102:0/1564886852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce8101870 0x7f0ce8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.993+0000 7f0cefc15640 1 --2- 172.21.15.102:0/1564886852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce8100680 0x7f0ce8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.993+0000 7f0cefc15640 1 -- 172.21.15.102:0/1564886852 >> 172.21.15.102:0/1564886852 conn(0x7f0ce80fbbf0 msgr2=0x7f0ce80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.993+0000 7f0cefc15640 1 -- 172.21.15.102:0/1564886852 shutdown_connections 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.993+0000 7f0cefc15640 1 -- 172.21.15.102:0/1564886852 wait complete. 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0cefc15640 1 Processor -- start 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0cefc15640 1 -- start start 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0cefc15640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce8100680 0x7f0ce81968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0cefc15640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8101870 0x7f0ce8196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0cefc15640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce8102230 0x7f0ce819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0cefc15640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810c190 con 0x7f0ce8101870 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0cefc15640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810c010 con 0x7f0ce8102230 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0cefc15640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810c310 con 0x7f0ce8100680 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced1b0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8101870 0x7f0ce8196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced1b0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8101870 0x7f0ce8196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53406/0 (socket says 172.21.15.102:53406) 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced1b0640 1 -- 172.21.15.102:0/2171444333 learned_addr learned my addr 172.21.15.102:0/2171444333 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced9b1640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce8100680 0x7f0ce81968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced1b0640 1 -- 172.21.15.102:0/2171444333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce8100680 msgr2=0x7f0ce81968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced1b0640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce8100680 0x7f0ce81968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced1b0640 1 -- 172.21.15.102:0/2171444333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce8102230 msgr2=0x7f0ce819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced1b0640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce8102230 0x7f0ce819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced1b0640 1 -- 172.21.15.102:0/2171444333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce819e4d0 con 0x7f0ce8101870 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.994+0000 7f0ced9b1640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce8100680 0x7f0ce81968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.995+0000 7f0ced1b0640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8101870 0x7f0ce8196df0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800c890 tx=0x7f0cd800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.995+0000 7f0cdeffd640 1 -- 172.21.15.102:0/2171444333 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8004220 con 0x7f0ce8101870 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.995+0000 7f0cdeffd640 1 -- 172.21.15.102:0/2171444333 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cd80043c0 con 0x7f0ce8101870 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.995+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce819e7c0 con 0x7f0ce8101870 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.995+0000 7f0cdeffd640 1 -- 172.21.15.102:0/2171444333 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8004a00 con 0x7f0ce8101870 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.995+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce819ed60 con 0x7f0ce8101870 2024-04-24T23:12:52.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.996+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce8107ac0 con 0x7f0ce8101870 2024-04-24T23:12:52.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.997+0000 7f0cdeffd640 1 -- 172.21.15.102:0/2171444333 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0cd8004ba0 con 0x7f0ce8101870 2024-04-24T23:12:52.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.997+0000 7f0cdeffd640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.997+0000 7f0ced9b1640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.997+0000 7f0cdeffd640 1 -- 172.21.15.102:0/2171444333 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f0cd80ced60 con 0x7f0ce8101870 2024-04-24T23:12:52.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:51.998+0000 7f0ced9b1640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ce81016d0 tx=0x7f0ce4009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:52.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.001+0000 7f0cdeffd640 1 -- 172.21.15.102:0/2171444333 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0cd8014030 con 0x7f0ce8101870 2024-04-24T23:12:52.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.176+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0ce802d170 con 0x7f0ce8101870 2024-04-24T23:12:52.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.177+0000 7f0cdeffd640 1 -- 172.21.15.102:0/2171444333 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f0cd809bc90 con 0x7f0ce8101870 2024-04-24T23:12:52.179 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-24T23:12:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.178+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cd40a2c10 msgr2=0x7f0cd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.178+0000 7f0cefc15640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ce81016d0 tx=0x7f0ce4009540 comp rx=0 tx=0).stop 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.178+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8101870 msgr2=0x7f0ce8196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.178+0000 7f0cefc15640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8101870 0x7f0ce8196df0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800c890 tx=0x7f0cd800cd50 comp rx=0 tx=0).stop 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.178+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 shutdown_connections 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.178+0000 7f0cefc15640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.178+0000 7f0cefc15640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ce8102230 0x7f0ce819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.178+0000 7f0cefc15640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ce8101870 0x7f0ce8196df0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.178+0000 7f0cefc15640 1 --2- 172.21.15.102:0/2171444333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce8100680 0x7f0ce81968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.179+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 >> 172.21.15.102:0/2171444333 conn(0x7f0ce80fbbf0 msgr2=0x7f0ce8104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.179+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 shutdown_connections 2024-04-24T23:12:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.179+0000 7f0cefc15640 1 -- 172.21.15.102:0/2171444333 wait complete. 2024-04-24T23:12:52.191 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:52.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 -- 172.21.15.102:0/3783486680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c103340 msgr2=0x7fb86c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 --2- 172.21.15.102:0/3783486680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c103340 0x7fb86c109bd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb864009900 tx=0x7fb86402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:52.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 -- 172.21.15.102:0/3783486680 shutdown_connections 2024-04-24T23:12:52.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 --2- 172.21.15.102:0/3783486680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c103340 0x7fb86c109bd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 --2- 172.21.15.102:0/3783486680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb86c102980 0x7fb86c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 --2- 172.21.15.102:0/3783486680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb86c101790 0x7fb86c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 -- 172.21.15.102:0/3783486680 >> 172.21.15.102:0/3783486680 conn(0x7fb86c0fcd20 msgr2=0x7fb86c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 -- 172.21.15.102:0/3783486680 shutdown_connections 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 -- 172.21.15.102:0/3783486680 wait complete. 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 Processor -- start 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 -- start start 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c101790 0x7fb86c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb86c102980 0x7fb86c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.291+0000 7fb87345d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb86c103340 0x7fb86c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb87345d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10d2a0 con 0x7fb86c101790 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb87345d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10d120 con 0x7fb86c102980 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb87345d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10d420 con 0x7fb86c103340 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8711f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c101790 0x7fb86c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8711f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c101790 0x7fb86c1978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53426/0 (socket says 172.21.15.102:53426) 2024-04-24T23:12:52.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8711f9640 1 -- 172.21.15.102:0/1230034241 learned_addr learned my addr 172.21.15.102:0/1230034241 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8709f8640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb86c102980 0x7fb86c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8719fa640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb86c103340 0x7fb86c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8711f9640 1 -- 172.21.15.102:0/1230034241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb86c103340 msgr2=0x7fb86c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8711f9640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb86c103340 0x7fb86c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8711f9640 1 -- 172.21.15.102:0/1230034241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb86c102980 msgr2=0x7fb86c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8711f9640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb86c102980 0x7fb86c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8711f9640 1 -- 172.21.15.102:0/1230034241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c19f510 con 0x7fb86c101790 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8709f8640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb86c102980 0x7fb86c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8711f9640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c101790 0x7fb86c1978f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb86800ccb0 tx=0x7fb86800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8627fc640 1 -- 172.21.15.102:0/1230034241 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868003990 con 0x7fb86c101790 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c19f800 con 0x7fb86c101790 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8627fc640 1 -- 172.21.15.102:0/1230034241 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb868004020 con 0x7fb86c101790 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb8627fc640 1 -- 172.21.15.102:0/1230034241 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86800f6b0 con 0x7fb86c101790 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.292+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c19fd50 con 0x7fb86c101790 2024-04-24T23:12:52.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.293+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c108ab0 con 0x7fb86c101790 2024-04-24T23:12:52.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.294+0000 7fb8627fc640 1 -- 172.21.15.102:0/1230034241 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb868003b30 con 0x7fb86c101790 2024-04-24T23:12:52.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.295+0000 7fb8627fc640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb8580a2b90 0x7fb8580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.295+0000 7fb8709f8640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb8580a2b90 0x7fb8580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.295+0000 7fb8627fc640 1 -- 172.21.15.102:0/1230034241 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb8680cf4f0 con 0x7fb86c101790 2024-04-24T23:12:52.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.295+0000 7fb8709f8640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb8580a2b90 0x7fb8580a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb86c198e00 tx=0x7fb85c00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:52.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.299+0000 7fb8627fc640 1 -- 172.21.15.102:0/1230034241 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb868014030 con 0x7fb86c101790 2024-04-24T23:12:52.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.474+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb86c02d170 con 0x7fb86c101790 2024-04-24T23:12:52.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.475+0000 7fb8627fc640 1 -- 172.21.15.102:0/1230034241 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fb86c02d170 con 0x7fb86c101790 2024-04-24T23:12:52.477 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.476+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb8580a2b90 msgr2=0x7fb8580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.476+0000 7fb87345d640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb8580a2b90 0x7fb8580a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb86c198e00 tx=0x7fb85c00a5c0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.476+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c101790 msgr2=0x7fb86c1978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.476+0000 7fb87345d640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c101790 0x7fb86c1978f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb86800ccb0 tx=0x7fb86800a620 comp rx=0 tx=0).stop 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.477+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 shutdown_connections 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.477+0000 7fb87345d640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb8580a2b90 0x7fb8580a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.477+0000 7fb87345d640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb86c103340 0x7fb86c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.477+0000 7fb87345d640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb86c102980 0x7fb86c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.477+0000 7fb87345d640 1 --2- 172.21.15.102:0/1230034241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb86c101790 0x7fb86c1978f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.477+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 >> 172.21.15.102:0/1230034241 conn(0x7fb86c0fcd20 msgr2=0x7fb86c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:52.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.477+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 shutdown_connections 2024-04-24T23:12:52.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.477+0000 7fb87345d640 1 -- 172.21.15.102:0/1230034241 wait complete. 2024-04-24T23:12:52.488 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:52 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4011819924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:52.488 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:52 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3204738571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:52.488 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:52 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2171444333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:52.490 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:52.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.584+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/2382502129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc100700 msgr2=0x7fa5cc100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.584+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/2382502129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc100700 0x7fa5cc100b20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009900 tx=0x7fa5c802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:52.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.584+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/2382502129 shutdown_connections 2024-04-24T23:12:52.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/2382502129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5cc1022b0 0x7fa5cc108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/2382502129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5cc1018f0 0x7fa5cc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/2382502129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc100700 0x7fa5cc100b20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/2382502129 >> 172.21.15.102:0/2382502129 conn(0x7fa5cc0fbc90 msgr2=0x7fa5cc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:52.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/2382502129 shutdown_connections 2024-04-24T23:12:52.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/2382502129 wait complete. 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 Processor -- start 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 -- start start 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5cc100700 0x7fa5cc197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5cc1018f0 0x7fa5cc197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc1022b0 0x7fa5cc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10c190 con 0x7fa5cc1022b0 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10c010 con 0x7fa5cc100700 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d3ab3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10c310 con 0x7fa5cc1018f0 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d2050640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc1022b0 0x7fa5cc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d2050640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc1022b0 0x7fa5cc19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53448/0 (socket says 172.21.15.102:53448) 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d2050640 1 -- 172.21.15.102:0/3797943037 learned_addr learned my addr 172.21.15.102:0/3797943037 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d104e640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5cc1018f0 0x7fa5cc197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.585+0000 7fa5d184f640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5cc100700 0x7fa5cc197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5d2050640 1 -- 172.21.15.102:0/3797943037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5cc1018f0 msgr2=0x7fa5cc197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5d2050640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5cc1018f0 0x7fa5cc197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5d2050640 1 -- 172.21.15.102:0/3797943037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5cc100700 msgr2=0x7fa5cc197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5d2050640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5cc100700 0x7fa5cc197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5d2050640 1 -- 172.21.15.102:0/3797943037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc19f520 con 0x7fa5cc1022b0 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5d2050640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc1022b0 0x7fa5cc19eeb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400eaa0 tx=0x7fa5c400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5beffd640 1 -- 172.21.15.102:0/3797943037 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4004d20 con 0x7fa5cc1022b0 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5beffd640 1 -- 172.21.15.102:0/3797943037 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5c4005020 con 0x7fa5cc1022b0 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc19f810 con 0x7fa5cc1022b0 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5beffd640 1 -- 172.21.15.102:0/3797943037 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40116b0 con 0x7fa5cc1022b0 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.586+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc19fd10 con 0x7fa5cc1022b0 2024-04-24T23:12:52.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.587+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc19ffd0 con 0x7fa5cc1022b0 2024-04-24T23:12:52.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.588+0000 7fa5beffd640 1 -- 172.21.15.102:0/3797943037 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa5c40051d0 con 0x7fa5cc1022b0 2024-04-24T23:12:52.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.588+0000 7fa5beffd640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5b80a2b90 0x7fa5b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.589+0000 7fa5d184f640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5b80a2b90 0x7fa5b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.589+0000 7fa5beffd640 1 -- 172.21.15.102:0/3797943037 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa5c40cf230 con 0x7fa5cc1022b0 2024-04-24T23:12:52.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.589+0000 7fa5d184f640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5b80a2b90 0x7fa5b80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8004770 tx=0x7fa5c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:52.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.593+0000 7fa5beffd640 1 -- 172.21.15.102:0/3797943037 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5c4016030 con 0x7fa5cc1022b0 2024-04-24T23:12:52.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:52 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4011819924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:52.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:52 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3204738571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:52.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:52 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2171444333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:52.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:52 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4011819924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:52.764 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:52 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3204738571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:52.764 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:52 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2171444333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:52.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.807+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5cc1a01a0 con 0x7fa5cc1022b0 2024-04-24T23:12:52.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.807+0000 7fa5beffd640 1 -- 172.21.15.102:0/3797943037 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7fa5c409c160 con 0x7fa5cc1022b0 2024-04-24T23:12:52.809 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:52.809 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:11:00.643113+0000","modified":"2024-04-24T23:11:04.856565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/1258581930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1258581930},{"type":"v1","addr":"172.21.15.161:6839","nonce":1258581930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24473,24497,24472]},"id":1}]} 2024-04-24T23:12:52.810 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-04-24T23:12:52.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5b80a2b90 msgr2=0x7fa5b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5b80a2b90 0x7fa5b80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8004770 tx=0x7fa5c8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc1022b0 msgr2=0x7fa5cc19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc1022b0 0x7fa5cc19eeb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400eaa0 tx=0x7fa5c400eea0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 shutdown_connections 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa5b80a2b90 0x7fa5b80a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5cc1022b0 0x7fa5cc19eeb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5cc1018f0 0x7fa5cc197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 --2- 172.21.15.102:0/3797943037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5cc100700 0x7fa5cc197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 >> 172.21.15.102:0/3797943037 conn(0x7fa5cc0fbc90 msgr2=0x7fa5cc0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 shutdown_connections 2024-04-24T23:12:52.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.809+0000 7fa5d3ab3640 1 -- 172.21.15.102:0/3797943037 wait complete. 2024-04-24T23:12:52.823 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:52.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.920+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/3887853845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 msgr2=0x7f2bd4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.920+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/3887853845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 0x7f2bd4100ac0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009900 tx=0x7f2bd002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.920+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/3887853845 shutdown_connections 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.920+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/3887853845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd4102250 0x7f2bd4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.920+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/3887853845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bd4101890 0x7f2bd4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.920+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/3887853845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 0x7f2bd4100ac0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.920+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/3887853845 >> 172.21.15.102:0/3887853845 conn(0x7f2bd40fbc30 msgr2=0x7f2bd40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.920+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/3887853845 shutdown_connections 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.920+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/3887853845 wait complete. 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdc5ee640 1 Processor -- start 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdc5ee640 1 -- start start 2024-04-24T23:12:52.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdc5ee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 0x7f2bd4199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdc5ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd4101890 0x7f2bd419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdc5ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bd4102250 0x7f2bd41a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdc5ee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd410b120 con 0x7f2bd41006a0 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdc5ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd410afa0 con 0x7f2bd4101890 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdc5ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd410b2a0 con 0x7f2bd4102250 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bda38a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 0x7f2bd4199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bda38a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 0x7f2bd4199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53462/0 (socket says 172.21.15.102:53462) 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bda38a640 1 -- 172.21.15.102:0/1856164485 learned_addr learned my addr 172.21.15.102:0/1856164485 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bd9b89640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd4101890 0x7f2bd419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdab8b640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bd4102250 0x7f2bd41a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bda38a640 1 -- 172.21.15.102:0/1856164485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bd4102250 msgr2=0x7f2bd41a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bda38a640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bd4102250 0x7f2bd41a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bda38a640 1 -- 172.21.15.102:0/1856164485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd4101890 msgr2=0x7f2bd419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bda38a640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd4101890 0x7f2bd419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bda38a640 1 -- 172.21.15.102:0/1856164485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd41a1790 con 0x7f2bd41006a0 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bdab8b640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bd4102250 0x7f2bd41a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.921+0000 7f2bda38a640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 0x7f2bd4199bc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00314e0 tx=0x7f2bd0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.922+0000 7f2bcb7fe640 1 -- 172.21.15.102:0/1856164485 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd003f070 con 0x7f2bd41006a0 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.922+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd41a1a20 con 0x7f2bd41006a0 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.922+0000 7f2bcb7fe640 1 -- 172.21.15.102:0/1856164485 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bd0031e40 con 0x7f2bd41006a0 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.922+0000 7f2bcb7fe640 1 -- 172.21.15.102:0/1856164485 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0035030 con 0x7f2bd41006a0 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.922+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd41a1f60 con 0x7f2bd41006a0 2024-04-24T23:12:52.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.922+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd4107a40 con 0x7f2bd41006a0 2024-04-24T23:12:52.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.924+0000 7f2bcb7fe640 1 -- 172.21.15.102:0/1856164485 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2bd0042070 con 0x7f2bd41006a0 2024-04-24T23:12:52.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.924+0000 7f2bcb7fe640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bc00a2b70 0x7f2bc00a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:52.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.924+0000 7f2bd9b89640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bc00a2b70 0x7f2bc00a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:52.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.924+0000 7f2bcb7fe640 1 -- 172.21.15.102:0/1856164485 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2bd00efa40 con 0x7f2bd41006a0 2024-04-24T23:12:52.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.925+0000 7f2bd9b89640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bc00a2b70 0x7f2bc00a5030 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bd419b080 tx=0x7f2bc400a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:52.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:52.928+0000 7f2bcb7fe640 1 -- 172.21.15.102:0/1856164485 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2bd00bc970 con 0x7f2bd41006a0 2024-04-24T23:12:53.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.107+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2bd4062340 con 0x7f2bd41006a0 2024-04-24T23:12:53.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.109+0000 7f2bcb7fe640 1 -- 172.21.15.102:0/1856164485 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22767 (secure 0 0 0) 0x7f2bd00bc270 con 0x7f2bd41006a0 2024-04-24T23:12:53.111 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:12:53.111 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":90,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:12:50.317275+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:53.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.110+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bc00a2b70 msgr2=0x7f2bc00a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.110+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bc00a2b70 0x7f2bc00a5030 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bd419b080 tx=0x7f2bc400a250 comp rx=0 tx=0).stop 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.110+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 msgr2=0x7f2bd4199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.110+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 0x7f2bd4199bc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00314e0 tx=0x7f2bd0003d50 comp rx=0 tx=0).stop 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.111+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 shutdown_connections 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.111+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f2bc00a2b70 0x7f2bc00a5030 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.111+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bd4102250 0x7f2bd41a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.111+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd4101890 0x7f2bd419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.111+0000 7f2bdc5ee640 1 --2- 172.21.15.102:0/1856164485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41006a0 0x7f2bd4199bc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.111+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 >> 172.21.15.102:0/1856164485 conn(0x7f2bd40fbc30 msgr2=0x7f2bd4104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.111+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 shutdown_connections 2024-04-24T23:12:53.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.111+0000 7f2bdc5ee640 1 -- 172.21.15.102:0/1856164485 wait complete. 2024-04-24T23:12:53.124 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T23:11:00.643113+0000', 'modified': '2024-04-24T23:11:04.856565+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24473, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/2266462643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 2266462643}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 2266462643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/2778927873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 2778927873}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 2778927873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/3933349751', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3933349751}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3933349751}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.183:6839/395274952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 395274952}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 395274952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.161:6837/2386140374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 2386140374}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 2386140374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/1258581930', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 1258581930}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 1258581930}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24473, 24497, 24472]} 2024-04-24T23:12:53.124 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T23:12:53.124 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:12:53.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.221+0000 7fb41e770640 1 -- 172.21.15.102:0/3392214723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418103340 msgr2=0x7fb418109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.221+0000 7fb41e770640 1 --2- 172.21.15.102:0/3392214723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418103340 0x7fb418109bd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb410009ca0 tx=0x7fb41002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.221+0000 7fb41e770640 1 -- 172.21.15.102:0/3392214723 shutdown_connections 2024-04-24T23:12:53.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.221+0000 7fb41e770640 1 --2- 172.21.15.102:0/3392214723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418103340 0x7fb418109bd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.221+0000 7fb41e770640 1 --2- 172.21.15.102:0/3392214723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb418102980 0x7fb418102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.221+0000 7fb41e770640 1 --2- 172.21.15.102:0/3392214723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418101790 0x7fb418101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.221+0000 7fb41e770640 1 -- 172.21.15.102:0/3392214723 >> 172.21.15.102:0/3392214723 conn(0x7fb4180fcd20 msgr2=0x7fb4180ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.221+0000 7fb41e770640 1 -- 172.21.15.102:0/3392214723 shutdown_connections 2024-04-24T23:12:53.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.221+0000 7fb41e770640 1 -- 172.21.15.102:0/3392214723 wait complete. 2024-04-24T23:12:53.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb41e770640 1 Processor -- start 2024-04-24T23:12:53.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb41e770640 1 -- start start 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb41e770640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418101790 0x7fb4181978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb41e770640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418102980 0x7fb418197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb41e770640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb418103340 0x7fb41819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb41e770640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810d2a0 con 0x7fb418102980 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb41e770640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810d120 con 0x7fb418103340 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb41e770640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810d420 con 0x7fb418101790 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418102980 0x7fb418197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418102980 0x7fb418197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53474/0 (socket says 172.21.15.102:53474) 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb4177fe640 1 -- 172.21.15.102:0/844112605 learned_addr learned my addr 172.21.15.102:0/844112605 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb41cd0d640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb418103340 0x7fb41819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb417fff640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418101790 0x7fb4181978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb4177fe640 1 -- 172.21.15.102:0/844112605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418101790 msgr2=0x7fb4181978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb4177fe640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418101790 0x7fb4181978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb4177fe640 1 -- 172.21.15.102:0/844112605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb418103340 msgr2=0x7fb41819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb4177fe640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb418103340 0x7fb41819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.222+0000 7fb4177fe640 1 -- 172.21.15.102:0/844112605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41819f500 con 0x7fb418102980 2024-04-24T23:12:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.223+0000 7fb4177fe640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418102980 0x7fb418197e20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c920 tx=0x7fb40c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.223+0000 7fb4157fa640 1 -- 172.21.15.102:0/844112605 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c007c70 con 0x7fb418102980 2024-04-24T23:12:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.223+0000 7fb41e770640 1 -- 172.21.15.102:0/844112605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41819f7f0 con 0x7fb418102980 2024-04-24T23:12:53.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.223+0000 7fb4157fa640 1 -- 172.21.15.102:0/844112605 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb40c007e10 con 0x7fb418102980 2024-04-24T23:12:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.223+0000 7fb4157fa640 1 -- 172.21.15.102:0/844112605 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c018820 con 0x7fb418102980 2024-04-24T23:12:53.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.223+0000 7fb41e770640 1 -- 172.21.15.102:0/844112605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41819fd90 con 0x7fb418102980 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb4157fa640 1 -- 172.21.15.102:0/844112605 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb40c0189c0 con 0x7fb418102980 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb421484740 1 Processor -- start 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb421484740 1 -- start start 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb421484740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2a440 0x5623d4f2a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb421484740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ae30 0x5623d4f2f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb421484740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb421484740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5623d4e8df00 con 0x5623d4f2ffa0 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb421484740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5623d4e36750 con 0x5623d4f2a440 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb421484740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5623d4e89ec0 con 0x5623d4f2ae30 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.225+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f32430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53480/0 (socket says 172.21.15.102:53480) 2024-04-24T23:12:53.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4177fe640 1 -- 172.21.15.102:0/1255499803 learned_addr learned my addr 172.21.15.102:0/1255499803 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb417fff640 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ae30 0x5623d4f2f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4157fa640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4000a2b90 0x7fb4000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4177fe640 1 -- 172.21.15.102:0/1255499803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ae30 msgr2=0x5623d4f2f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4177fe640 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ae30 0x5623d4f2f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4177fe640 1 -- 172.21.15.102:0/1255499803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2a440 msgr2=0x5623d4f2a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb41cd0d640 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2a440 0x5623d4f2a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4177fe640 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2a440 0x5623d4f2a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4177fe640 1 -- 172.21.15.102:0/1255499803 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d4ba22d0 con 0x5623d4f2ffa0 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb417fff640 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ae30 0x5623d4f2f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb41cd0d640 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2a440 0x5623d4f2a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb41cd0d640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4000a2b90 0x7fb4000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4157fa640 1 -- 172.21.15.102:0/844112605 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb40c0cf2f0 con 0x7fb418102980 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb4177fe640 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f32430 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0be6d0 tx=0x7fb40c0c6650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9181ee8c8ca33ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb3fe7fc640 1 -- 172.21.15.102:0/1255499803 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c09b890 con 0x5623d4f2ffa0 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb3fe7fc640 1 -- 172.21.15.102:0/1255499803 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb40c00fb00 con 0x5623d4f2ffa0 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.226+0000 7fb3fe7fc640 1 -- 172.21.15.102:0/1255499803 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c018cb0 con 0x5623d4f2ffa0 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 -- 172.21.15.102:0/1255499803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 msgr2=0x5623d4f32430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb41cd0d640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4000a2b90 0x7fb4000a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5623d4f2b690 tx=0x7fb4100095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f32430 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0be6d0 tx=0x7fb40c0c6650 comp rx=0 tx=0).stop 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 -- 172.21.15.102:0/1255499803 shutdown_connections 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f32430 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ae30 0x5623d4f2f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 --2- 172.21.15.102:0/1255499803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2a440 0x5623d4f2a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 -- 172.21.15.102:0/1255499803 >> 172.21.15.102:0/1255499803 conn(0x5623d4f25ab0 msgr2=0x5623d4f25eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 -- 172.21.15.102:0/1255499803 shutdown_connections 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 -- 172.21.15.102:0/1255499803 wait complete. 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 -- start start 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.227+0000 7fb421484740 1 Processor -- start 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2a440 0x5623d4f347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 0x5623d4f34ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ffa0 0x5623d4f3bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5623d4f32ef0 con 0x5623d4f2a440 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5623d4e49cf0 con 0x5623d4f2ae30 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5623d4f33070 con 0x5623d4f2ffa0 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 10 client.0 _ll_register_callbacks cb 0x5623d4e496e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb417fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2a440 0x5623d4f347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb417fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2a440 0x5623d4f347a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53486/0 (socket says 172.21.15.102:53486) 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb417fff640 1 -- 172.21.15.102:0/3972752847 learned_addr learned my addr 172.21.15.102:0/3972752847 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb4177fe640 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 0x5623d4f34ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb417fff640 1 -- 172.21.15.102:0/3972752847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ffa0 msgr2=0x5623d4f3bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb417fff640 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ffa0 0x5623d4f3bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb417fff640 1 -- 172.21.15.102:0/3972752847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 msgr2=0x5623d4f34ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb417fff640 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 0x5623d4f34ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb417fff640 1 -- 172.21.15.102:0/3972752847 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623d4c6c6b0 con 0x5623d4f2a440 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb417fff640 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2a440 0x5623d4f347a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb40800af60 tx=0x7fb40800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d278eec2bd1060 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb3f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb3f77fe640 1 -- 172.21.15.102:0/3972752847 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408011850 con 0x5623d4f2a440 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb3f77fe640 1 -- 172.21.15.102:0/3972752847 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb4080119f0 con 0x5623d4f2a440 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 1 -- 172.21.15.102:0/3972752847 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623d4e9b440 con 0x5623d4f2a440 2024-04-24T23:12:53.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb3f77fe640 1 -- 172.21.15.102:0/3972752847 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb408011bf0 con 0x5623d4f2a440 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 10 client.15222 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.228+0000 7fb421484740 1 -- 172.21.15.102:0/3972752847 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5623d4f3ed40 con 0x5623d4f2a440 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.229+0000 7fb3f77fe640 1 -- 172.21.15.102:0/3972752847 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5623d4e9b440 con 0x5623d4f2a440 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.229+0000 7fb3f77fe640 1 -- 172.21.15.102:0/3972752847 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb408023070 con 0x5623d4f2a440 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.229+0000 7fb421484740 10 client.15222 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.229+0000 7fb421484740 10 client.15222 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.229+0000 7fb421484740 20 client.15222 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.229+0000 7fb421484740 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x5623d4f41430 0x5623d4f428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.229+0000 7fb421484740 4 client.15222 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.229+0000 7fb421484740 1 -- 172.21.15.102:0/3972752847 --> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623d4e95790 con 0x5623d4f41430 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.229+0000 7fb4177fe640 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x5623d4f41430 0x5623d4f428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb3f77fe640 10 client.15222 ms_handle_connect on v2:172.21.15.183:6838/395274952 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb4177fe640 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x5623d4f41430 0x5623d4f428a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9e56cf3631588e2 server_cookie=25903ada29cf3ad8 in_seq=0 out_seq=0 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb3f77fe640 1 -- 172.21.15.102:0/3972752847 <== mds.2 v2:172.21.15.183:6838/395274952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623d4e95790 con 0x5623d4f41430 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb3f77fe640 10 client.15222 handle_command_reply: tid=0 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb421484740 1 client.15222 shutdown 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb421484740 1 -- 172.21.15.102:0/3972752847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2a440 msgr2=0x5623d4f347a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb421484740 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2a440 0x5623d4f347a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb40800af60 tx=0x7fb40800af90 comp rx=0 tx=0).stop 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb421484740 1 -- 172.21.15.102:0/3972752847 shutdown_connections 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb421484740 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x5623d4f41430 0x5623d4f428a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb421484740 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2ffa0 0x5623d4f3bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb3f77fe640 0 client.15222 ms_handle_reset on v2:172.21.15.183:6838/395274952 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb421484740 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 0x5623d4f34ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb421484740 1 --2- 172.21.15.102:0/3972752847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2a440 0x5623d4f347a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.230+0000 7fb421484740 1 -- 172.21.15.102:0/3972752847 >> 172.21.15.102:0/3972752847 conn(0x5623d4f25ab0 msgr2=0x5623d4f248f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.231+0000 7fb421484740 1 -- 172.21.15.102:0/3972752847 shutdown_connections 2024-04-24T23:12:53.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.231+0000 7fb421484740 1 -- 172.21.15.102:0/3972752847 wait complete. 2024-04-24T23:12:53.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.231+0000 7fb421484740 20 client.15222 trim_cache size 0 max 16384 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.248+0000 7fb421484740 1 Processor -- start 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.248+0000 7fb421484740 1 -- start start 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb421484740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 0x5623d4f20690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb421484740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ae30 0x5623d4f3bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb421484740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ffa0 0x5623d4f23fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb421484740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5623d4f40be0 con 0x5623d4f2ae30 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb421484740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5623d4f412b0 con 0x5623d4f2ffa0 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb421484740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5623d4f41980 con 0x5623d4f2a440 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ae30 0x5623d4f3bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ae30 0x5623d4f3bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53502/0 (socket says 172.21.15.102:53502) 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb4177fe640 1 -- 172.21.15.102:0/1914281266 learned_addr learned my addr 172.21.15.102:0/1914281266 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb4177fe640 1 -- 172.21.15.102:0/1914281266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 msgr2=0x5623d4f20690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb4177fe640 1 --2- 172.21.15.102:0/1914281266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 0x5623d4f20690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb4177fe640 1 -- 172.21.15.102:0/1914281266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ffa0 msgr2=0x5623d4f23fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb4177fe640 1 --2- 172.21.15.102:0/1914281266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ffa0 0x5623d4f23fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb4177fe640 1 -- 172.21.15.102:0/1914281266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d4f258e0 con 0x5623d4f2ae30 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb4177fe640 1 --2- 172.21.15.102:0/1914281266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ae30 0x5623d4f3bc40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0a4200 tx=0x7fb40c0d2e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bad65ba1b2ff095 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb3fdffb640 1 -- 172.21.15.102:0/1914281266 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0be530 con 0x5623d4f2ae30 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb3fdffb640 1 -- 172.21.15.102:0/1914281266 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb40c0d2b00 con 0x5623d4f2ae30 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.249+0000 7fb3fdffb640 1 -- 172.21.15.102:0/1914281266 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c09b420 con 0x5623d4f2ae30 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 -- 172.21.15.102:0/1914281266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ae30 msgr2=0x5623d4f3bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 --2- 172.21.15.102:0/1914281266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ae30 0x5623d4f3bc40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0a4200 tx=0x7fb40c0d2e20 comp rx=0 tx=0).stop 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 -- 172.21.15.102:0/1914281266 shutdown_connections 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 --2- 172.21.15.102:0/1914281266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ffa0 0x5623d4f23fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 --2- 172.21.15.102:0/1914281266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ae30 0x5623d4f3bc40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 --2- 172.21.15.102:0/1914281266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 0x5623d4f20690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 -- 172.21.15.102:0/1914281266 >> 172.21.15.102:0/1914281266 conn(0x5623d4f25ab0 msgr2=0x5623d4f25e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 -- 172.21.15.102:0/1914281266 shutdown_connections 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 -- 172.21.15.102:0/1914281266 wait complete. 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 -- start start 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.250+0000 7fb421484740 1 Processor -- start 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 0x5623d4f2e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 0x5623d4f2e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f3a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5623d4f42720 con 0x5623d4f2ffa0 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5623d4f42050 con 0x5623d4f2ae30 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5623d4f42df0 con 0x5623d4f2a440 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 10 client.0 _ll_register_callbacks cb 0x7fb40800aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb41cd0d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f3a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb41cd0d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f3a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53514/0 (socket says 172.21.15.102:53514) 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb41cd0d640 1 -- 172.21.15.102:0/2650454762 learned_addr learned my addr 172.21.15.102:0/2650454762 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb417fff640 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 0x5623d4f2e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb4177fe640 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 0x5623d4f2e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb41cd0d640 1 -- 172.21.15.102:0/2650454762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 msgr2=0x5623d4f2e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb41cd0d640 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 0x5623d4f2e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb41cd0d640 1 -- 172.21.15.102:0/2650454762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 msgr2=0x5623d4f2e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb41cd0d640 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 0x5623d4f2e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb41cd0d640 1 -- 172.21.15.102:0/2650454762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623d4f29fa0 con 0x5623d4f2ffa0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb417fff640 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 0x5623d4f2e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb41cd0d640 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f3a330 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb410035000 tx=0x7fb410003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1eaabed03d3c3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb3f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb3f77fe640 1 -- 172.21.15.102:0/2650454762 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4100305c0 con 0x5623d4f2ffa0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb3f77fe640 1 -- 172.21.15.102:0/2650454762 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb410030760 con 0x5623d4f2ffa0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 1 -- 172.21.15.102:0/2650454762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40c0d2b00 con 0x5623d4f2ffa0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb3f77fe640 1 -- 172.21.15.102:0/2650454762 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb410030960 con 0x5623d4f2ffa0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 10 client.15228 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.251+0000 7fb421484740 1 -- 172.21.15.102:0/2650454762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5623d4f3ba50 con 0x5623d4f2ffa0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb3f77fe640 1 -- 172.21.15.102:0/2650454762 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb40c0d2b00 con 0x5623d4f2ffa0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb3f77fe640 1 -- 172.21.15.102:0/2650454762 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb410043040 con 0x5623d4f2ffa0 2024-04-24T23:12:53.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb421484740 10 client.15228 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb421484740 10 client.15228 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb421484740 20 client.15228 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb421484740 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x5623d4f41430 0x5623d4faa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb421484740 4 client.15228 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb421484740 1 -- 172.21.15.102:0/2650454762 --> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] -- command(tid 0: {"prefix": "status"}) -- 0x5623d4e13b80 con 0x5623d4f41430 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb417fff640 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x5623d4f41430 0x5623d4faa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb3f77fe640 10 client.15228 ms_handle_connect on v2:172.21.15.183:6838/395274952 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.252+0000 7fb417fff640 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x5623d4f41430 0x5623d4faa630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a932e6d074d62484 server_cookie=cdbc1aaeba0a6576 in_seq=0 out_seq=0 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb3f77fe640 1 -- 172.21.15.102:0/2650454762 <== mds.2 v2:172.21.15.183:6838/395274952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5623d4e13b80 con 0x5623d4f41430 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb3f77fe640 10 client.15228 handle_command_reply: tid=0 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 client.15228 shutdown 2024-04-24T23:12:53.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 -- 172.21.15.102:0/2650454762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 msgr2=0x5623d4f3a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f3a330 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb410035000 tx=0x7fb410003d40 comp rx=0 tx=0).stop 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 -- 172.21.15.102:0/2650454762 shutdown_connections 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.183:6838/395274952,v1:172.21.15.183:6839/395274952] conn(0x5623d4f41430 0x5623d4faa630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623d4f2ffa0 0x5623d4f3a330 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623d4f2ae30 0x5623d4f2e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb3f77fe640 0 client.15228 ms_handle_reset on v2:172.21.15.183:6838/395274952 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 --2- 172.21.15.102:0/2650454762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623d4f2a440 0x5623d4f2e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 -- 172.21.15.102:0/2650454762 >> 172.21.15.102:0/2650454762 conn(0x5623d4f25ab0 msgr2=0x5623d4f20690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 -- 172.21.15.102:0/2650454762 shutdown_connections 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 1 -- 172.21.15.102:0/2650454762 wait complete. 2024-04-24T23:12:53.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb421484740 20 client.15228 trim_cache size 0 max 16384 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 2, 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 24472, 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 108.919608238, 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 20, 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 90, 2024-04-24T23:12:53.264 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T23:12:53.265 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 126.47739632299999 2024-04-24T23:12:53.265 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:12:53.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb41e770640 1 -- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4000a2b90 msgr2=0x7fb4000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb41e770640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4000a2b90 0x7fb4000a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5623d4f2b690 tx=0x7fb4100095c0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb41e770640 1 -- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418102980 msgr2=0x7fb418197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb41e770640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418102980 0x7fb418197e20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c920 tx=0x7fb40c00cde0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb41e770640 1 -- 172.21.15.102:0/844112605 shutdown_connections 2024-04-24T23:12:53.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb41e770640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fb4000a2b90 0x7fb4000a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb41e770640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb418103340 0x7fb41819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb41e770640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb418102980 0x7fb418197e20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.253+0000 7fb41e770640 1 --2- 172.21.15.102:0/844112605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418101790 0x7fb4181978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.254+0000 7fb41e770640 1 -- 172.21.15.102:0/844112605 >> 172.21.15.102:0/844112605 conn(0x7fb4180fcd20 msgr2=0x7fb418105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.254+0000 7fb41e770640 1 -- 172.21.15.102:0/844112605 shutdown_connections 2024-04-24T23:12:53.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.254+0000 7fb41e770640 1 -- 172.21.15.102:0/844112605 wait complete. 2024-04-24T23:12:53.267 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-24T23:12:53.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.365+0000 7f30599d5640 1 -- 172.21.15.102:0/114393070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 msgr2=0x7f3054102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.365+0000 7f30599d5640 1 --2- 172.21.15.102:0/114393070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 0x7f3054102e00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3048009900 tx=0x7f304802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:53.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.365+0000 7f30599d5640 1 -- 172.21.15.102:0/114393070 shutdown_connections 2024-04-24T23:12:53.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.365+0000 7f30599d5640 1 --2- 172.21.15.102:0/114393070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3054103340 0x7f3054109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.365+0000 7f30599d5640 1 --2- 172.21.15.102:0/114393070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 0x7f3054102e00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.365+0000 7f30599d5640 1 --2- 172.21.15.102:0/114393070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3054101790 0x7f3054101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.365+0000 7f30599d5640 1 -- 172.21.15.102:0/114393070 >> 172.21.15.102:0/114393070 conn(0x7f30540fcd20 msgr2=0x7f30540ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.365+0000 7f30599d5640 1 -- 172.21.15.102:0/114393070 shutdown_connections 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.365+0000 7f30599d5640 1 -- 172.21.15.102:0/114393070 wait complete. 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30599d5640 1 Processor -- start 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30599d5640 1 -- start start 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30599d5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3054101790 0x7f3054197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30599d5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 0x7f3054197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30599d5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3054103340 0x7f305419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30599d5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f305410d320 con 0x7f3054102980 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30599d5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f305410d1a0 con 0x7f3054103340 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30599d5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f305410d4a0 con 0x7f3054101790 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30527fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 0x7f3054197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30527fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 0x7f3054197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53530/0 (socket says 172.21.15.102:53530) 2024-04-24T23:12:53.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30527fc640 1 -- 172.21.15.102:0/1580982572 learned_addr learned my addr 172.21.15.102:0/1580982572 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30537fe640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3054103340 0x7f305419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f3052ffd640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3054101790 0x7f3054197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30527fc640 1 -- 172.21.15.102:0/1580982572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3054101790 msgr2=0x7f3054197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30527fc640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3054101790 0x7f3054197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30527fc640 1 -- 172.21.15.102:0/1580982572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3054103340 msgr2=0x7f305419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30527fc640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3054103340 0x7f305419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30527fc640 1 -- 172.21.15.102:0/1580982572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305419f670 con 0x7f3054102980 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30537fe640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3054103340 0x7f305419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f3052ffd640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3054101790 0x7f3054197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.366+0000 7f30527fc640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 0x7f3054197f90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3048004770 tx=0x7f30480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.367+0000 7f30589d3640 1 -- 172.21.15.102:0/1580982572 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304803f030 con 0x7f3054102980 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.367+0000 7f30589d3640 1 -- 172.21.15.102:0/1580982572 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3048003cb0 con 0x7f3054102980 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.367+0000 7f30599d5640 1 -- 172.21.15.102:0/1580982572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305419f900 con 0x7f3054102980 2024-04-24T23:12:53.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.367+0000 7f30589d3640 1 -- 172.21.15.102:0/1580982572 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048035070 con 0x7f3054102980 2024-04-24T23:12:53.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.367+0000 7f30599d5640 1 -- 172.21.15.102:0/1580982572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305419fe70 con 0x7f3054102980 2024-04-24T23:12:53.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f30589d3640 1 -- 172.21.15.102:0/1580982572 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3048042030 con 0x7f3054102980 2024-04-24T23:12:53.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f305c7c1740 1 Processor -- start 2024-04-24T23:12:53.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f305c7c1740 1 -- start start 2024-04-24T23:12:53.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963bfa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963bff930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfffa0 0x557963c02430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f305c7c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557963b5df00 con 0x557963bfae30 2024-04-24T23:12:53.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f305c7c1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557963b06750 con 0x557963bfa440 2024-04-24T23:12:53.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f305c7c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557963b59ec0 con 0x557963bfffa0 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963bff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963bff930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53538/0 (socket says 172.21.15.102:53538) 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f3052ffd640 1 -- 172.21.15.102:0/949061411 learned_addr learned my addr 172.21.15.102:0/949061411 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f30589d3640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f303c0a2b90 0x7f303c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f30527fc640 1 --2- 172.21.15.102:0/949061411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfffa0 0x557963c02430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.369+0000 7f30537fe640 1 --2- 172.21.15.102:0/949061411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963bfa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f3052ffd640 1 -- 172.21.15.102:0/949061411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfffa0 msgr2=0x557963c02430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f30537fe640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f303c0a2b90 0x7f303c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f3052ffd640 1 --2- 172.21.15.102:0/949061411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfffa0 0x557963c02430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f3052ffd640 1 -- 172.21.15.102:0/949061411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 msgr2=0x557963bfa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f3052ffd640 1 --2- 172.21.15.102:0/949061411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963bfa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f3052ffd640 1 -- 172.21.15.102:0/949061411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579638722d0 con 0x557963bfae30 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f30589d3640 1 -- 172.21.15.102:0/1580982572 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f30480efab0 con 0x7f3054102980 2024-04-24T23:12:53.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f3052ffd640 1 --2- 172.21.15.102:0/949061411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963bff930 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3044009f10 tx=0x7f304400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbbc9a39c779382 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f3039ffb640 1 -- 172.21.15.102:0/949061411 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304401a680 con 0x557963bfae30 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f30537fe640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f303c0a2b90 0x7f303c0a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x557963bfb690 tx=0x7f304c006da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f3039ffb640 1 -- 172.21.15.102:0/949061411 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30440118f0 con 0x557963bfae30 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.370+0000 7f3039ffb640 1 -- 172.21.15.102:0/949061411 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044010890 con 0x557963bfae30 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 -- 172.21.15.102:0/949061411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 msgr2=0x557963bff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 --2- 172.21.15.102:0/949061411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963bff930 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3044009f10 tx=0x7f304400f730 comp rx=0 tx=0).stop 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 -- 172.21.15.102:0/949061411 shutdown_connections 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 --2- 172.21.15.102:0/949061411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfffa0 0x557963c02430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 --2- 172.21.15.102:0/949061411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963bff930 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 --2- 172.21.15.102:0/949061411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963bfa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 -- 172.21.15.102:0/949061411 >> 172.21.15.102:0/949061411 conn(0x557963bf5ab0 msgr2=0x557963bf5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 -- 172.21.15.102:0/949061411 shutdown_connections 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 -- 172.21.15.102:0/949061411 wait complete. 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 -- start start 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 Processor -- start 2024-04-24T23:12:53.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfa440 0x557963c04890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfae30 0x557963c04dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963c0be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557963c02f50 con 0x557963bfffa0 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557963b19cf0 con 0x557963bfae30 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557963c030d0 con 0x557963bfa440 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.371+0000 7f305c7c1740 10 client.0 _ll_register_callbacks cb 0x557963b196e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30537fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963c0be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30537fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963c0be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53554/0 (socket says 172.21.15.102:53554) 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30537fe640 1 -- 172.21.15.102:0/1423896209 learned_addr learned my addr 172.21.15.102:0/1423896209 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f3052ffd640 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfa440 0x557963c04890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30527fc640 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfae30 0x557963c04dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30537fe640 1 -- 172.21.15.102:0/1423896209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfa440 msgr2=0x557963c04890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30537fe640 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfa440 0x557963c04890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30537fe640 1 -- 172.21.15.102:0/1423896209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfae30 msgr2=0x557963c04dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30537fe640 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfae30 0x557963c04dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30537fe640 1 -- 172.21.15.102:0/1423896209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55796393c6b0 con 0x557963bfffa0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30527fc640 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfae30 0x557963c04dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f30537fe640 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963c0be10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f304c012ff0 tx=0x7f304c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe6b677a8124431 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f3032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f3032ffd640 1 -- 172.21.15.102:0/1423896209 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c01f580 con 0x557963bfffa0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f3032ffd640 1 -- 172.21.15.102:0/1423896209 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f304c01f720 con 0x557963bfffa0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f305c7c1740 1 -- 172.21.15.102:0/1423896209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30440118f0 con 0x557963bfffa0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f3032ffd640 1 -- 172.21.15.102:0/1423896209 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f304c01f920 con 0x557963bfffa0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f305c7c1740 10 client.15240 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f305c7c1740 1 -- 172.21.15.102:0/1423896209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557963c0ee60 con 0x557963bfffa0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f3032ffd640 1 -- 172.21.15.102:0/1423896209 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30440118f0 con 0x557963bfffa0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.372+0000 7f3032ffd640 1 -- 172.21.15.102:0/1423896209 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f304c020a30 con 0x557963bfffa0 2024-04-24T23:12:53.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.373+0000 7f305c7c1740 10 client.15240 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.373+0000 7f305c7c1740 10 client.15240 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.373+0000 7f305c7c1740 20 client.15240 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.373+0000 7f305c7c1740 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x557963c115e0 0x557963c12a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.373+0000 7f305c7c1740 4 client.15240 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.373+0000 7f305c7c1740 1 -- 172.21.15.102:0/1423896209 --> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557963b6b440 con 0x557963c115e0 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.373+0000 7f3052ffd640 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x557963c115e0 0x557963c12a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.373+0000 7f3032ffd640 10 client.15240 ms_handle_connect on v2:172.21.15.161:6836/2386140374 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.373+0000 7f3052ffd640 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x557963c115e0 0x557963c12a50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2592dcdf03b0fcd server_cookie=84a98575184dc671 in_seq=0 out_seq=0 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f3032ffd640 1 -- 172.21.15.102:0/1423896209 <== mds.1 v2:172.21.15.161:6836/2386140374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557963b6b440 con 0x557963c115e0 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f3032ffd640 10 client.15240 handle_command_reply: tid=0 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 client.15240 shutdown 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 -- 172.21.15.102:0/1423896209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 msgr2=0x557963c0be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963c0be10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f304c012ff0 tx=0x7f304c0047d0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 -- 172.21.15.102:0/1423896209 shutdown_connections 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x557963c115e0 0x557963c12a50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963c0be10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfae30 0x557963c04dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 --2- 172.21.15.102:0/1423896209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfa440 0x557963c04890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f3032ffd640 0 client.15240 ms_handle_reset on v2:172.21.15.161:6836/2386140374 2024-04-24T23:12:53.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 -- 172.21.15.102:0/1423896209 >> 172.21.15.102:0/1423896209 conn(0x557963bf5ab0 msgr2=0x557963bf4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 -- 172.21.15.102:0/1423896209 shutdown_connections 2024-04-24T23:12:53.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 1 -- 172.21.15.102:0/1423896209 wait complete. 2024-04-24T23:12:53.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.374+0000 7f305c7c1740 20 client.15240 trim_cache size 0 max 16384 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f305c7c1740 1 Processor -- start 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f305c7c1740 1 -- start start 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963c02390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfae30 0x557963c0bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963bf5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f305c7c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557963c10d90 con 0x557963bfffa0 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f305c7c1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557963c11460 con 0x557963bfa440 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f305c7c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557963c11b30 con 0x557963bfae30 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f30537fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963bf5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f30537fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963bf5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53562/0 (socket says 172.21.15.102:53562) 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f30537fe640 1 -- 172.21.15.102:0/3360932708 learned_addr learned my addr 172.21.15.102:0/3360932708 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f30537fe640 1 -- 172.21.15.102:0/3360932708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfae30 msgr2=0x557963c0bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f30537fe640 1 --2- 172.21.15.102:0/3360932708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfae30 0x557963c0bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f30537fe640 1 -- 172.21.15.102:0/3360932708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 msgr2=0x557963c02390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f3052ffd640 1 --2- 172.21.15.102:0/3360932708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963c02390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f30537fe640 1 --2- 172.21.15.102:0/3360932708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963c02390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f30537fe640 1 -- 172.21.15.102:0/3360932708 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557963bfb6c0 con 0x557963bfffa0 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f3052ffd640 1 --2- 172.21.15.102:0/3360932708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963c02390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.392+0000 7f30537fe640 1 --2- 172.21.15.102:0/3360932708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963bf5040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f304c007f80 tx=0x7f304c007fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be077ac3c1fe98c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f30397fa640 1 -- 172.21.15.102:0/3360932708 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c039070 con 0x557963bfffa0 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f30397fa640 1 -- 172.21.15.102:0/3360932708 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f304c025070 con 0x557963bfffa0 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f30397fa640 1 -- 172.21.15.102:0/3360932708 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c0208b0 con 0x557963bfffa0 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 -- 172.21.15.102:0/3360932708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 msgr2=0x557963bf5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3360932708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963bf5040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f304c007f80 tx=0x7f304c007fb0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 -- 172.21.15.102:0/3360932708 shutdown_connections 2024-04-24T23:12:53.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3360932708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfffa0 0x557963bf5040 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3360932708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfae30 0x557963c0bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3360932708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963c02390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 -- 172.21.15.102:0/3360932708 >> 172.21.15.102:0/3360932708 conn(0x557963bf5ab0 msgr2=0x557963c04890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 -- 172.21.15.102:0/3360932708 shutdown_connections 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 -- 172.21.15.102:0/3360932708 wait complete. 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.393+0000 7f305c7c1740 1 -- start start 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f305c7c1740 1 Processor -- start 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963c08000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963c08540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f305c7c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfffa0 0x557963c0af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f305c7c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557963c128d0 con 0x557963bfae30 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f305c7c1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557963c12200 con 0x557963bfa440 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f305c7c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557963c12fa0 con 0x557963bfffa0 2024-04-24T23:12:53.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f305c7c1740 10 client.0 _ll_register_callbacks cb 0x7f304c012c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f30527fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963c08540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f30527fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963c08540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53564/0 (socket says 172.21.15.102:53564) 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f30527fc640 1 -- 172.21.15.102:0/3038535342 learned_addr learned my addr 172.21.15.102:0/3038535342 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f30527fc640 1 -- 172.21.15.102:0/3038535342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfffa0 msgr2=0x557963c0af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f30527fc640 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfffa0 0x557963c0af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f30527fc640 1 -- 172.21.15.102:0/3038535342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 msgr2=0x557963c08000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f30527fc640 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963c08000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f30527fc640 1 -- 172.21.15.102:0/3038535342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557963c0b540 con 0x557963bfae30 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f30527fc640 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963c08540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f30480314e0 tx=0x7f30480bc170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187bb3326074fedd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f3032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f3032ffd640 1 -- 172.21.15.102:0/3038535342 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048049040 con 0x557963bfae30 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f3032ffd640 1 -- 172.21.15.102:0/3038535342 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f304803bc90 con 0x557963bfae30 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.394+0000 7f305c7c1740 1 -- 172.21.15.102:0/3038535342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f304c025070 con 0x557963bfae30 2024-04-24T23:12:53.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f3032ffd640 1 -- 172.21.15.102:0/3038535342 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f30480bc9e0 con 0x557963bfae30 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f305c7c1740 10 client.15246 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f305c7c1740 1 -- 172.21.15.102:0/3038535342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557963c77a30 con 0x557963bfae30 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f3032ffd640 1 -- 172.21.15.102:0/3038535342 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f304c025070 con 0x557963bfae30 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f3032ffd640 1 -- 172.21.15.102:0/3038535342 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f30480d5140 con 0x557963bfae30 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f305c7c1740 10 client.15246 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f305c7c1740 10 client.15246 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f305c7c1740 20 client.15246 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x557963c7c710 0x557963c7b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f305c7c1740 4 client.15246 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f305c7c1740 1 -- 172.21.15.102:0/3038535342 --> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] -- command(tid 0: {"prefix": "status"}) -- 0x557963b65790 con 0x557963c7c710 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f3052ffd640 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x557963c7c710 0x557963c7b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f3032ffd640 10 client.15246 ms_handle_connect on v2:172.21.15.161:6836/2386140374 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.395+0000 7f3052ffd640 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x557963c7c710 0x557963c7b420 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=792755c2a64357e3 server_cookie=a13d00aad2fc235d in_seq=0 out_seq=0 2024-04-24T23:12:53.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f3032ffd640 1 -- 172.21.15.102:0/3038535342 <== mds.1 v2:172.21.15.161:6836/2386140374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557963b65790 con 0x557963c7c710 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f3032ffd640 10 client.15246 handle_command_reply: tid=0 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 client.15246 shutdown 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 -- 172.21.15.102:0/3038535342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 msgr2=0x557963c08540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963c08540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f30480314e0 tx=0x7f30480bc170 comp rx=0 tx=0).stop 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 -- 172.21.15.102:0/3038535342 shutdown_connections 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.161:6836/2386140374,v1:172.21.15.161:6837/2386140374] conn(0x557963c7c710 0x557963c7b420 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557963bfffa0 0x557963c0af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557963bfae30 0x557963c08540 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f3032ffd640 0 client.15246 ms_handle_reset on v2:172.21.15.161:6836/2386140374 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 --2- 172.21.15.102:0/3038535342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557963bfa440 0x557963c08000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 -- 172.21.15.102:0/3038535342 >> 172.21.15.102:0/3038535342 conn(0x557963bf5ab0 msgr2=0x557963c03b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 -- 172.21.15.102:0/3038535342 shutdown_connections 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 1 -- 172.21.15.102:0/3038535342 wait complete. 2024-04-24T23:12:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f305c7c1740 20 client.15246 trim_cache size 0 max 16384 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 1, 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 24473, 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 110.073944834, 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 20, 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 90, 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 129.38355617100001 2024-04-24T23:12:53.405 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:12:53.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f30599d5640 1 -- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f303c0a2b90 msgr2=0x7f303c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f30599d5640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f303c0a2b90 0x7f303c0a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x557963bfb690 tx=0x7f304c006da0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f30599d5640 1 -- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 msgr2=0x7f3054197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f30599d5640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 0x7f3054197f90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3048004770 tx=0x7f30480042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.396+0000 7f30599d5640 1 -- 172.21.15.102:0/1580982572 shutdown_connections 2024-04-24T23:12:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.397+0000 7f30599d5640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f303c0a2b90 0x7f303c0a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.397+0000 7f30599d5640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3054103340 0x7f305419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.397+0000 7f30599d5640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3054102980 0x7f3054197f90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.397+0000 7f30599d5640 1 --2- 172.21.15.102:0/1580982572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3054101790 0x7f3054197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.397+0000 7f30599d5640 1 -- 172.21.15.102:0/1580982572 >> 172.21.15.102:0/1580982572 conn(0x7f30540fcd20 msgr2=0x7f3054105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.397+0000 7f30599d5640 1 -- 172.21.15.102:0/1580982572 shutdown_connections 2024-04-24T23:12:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.397+0000 7f30599d5640 1 -- 172.21.15.102:0/1580982572 wait complete. 2024-04-24T23:12:53.408 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:53 smithi102 ceph-mon[29216]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T23:12:53.408 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:53 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1230034241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:53.408 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:53 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3797943037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:53.408 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:53 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1856164485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:53.410 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:12:53.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 -- 172.21.15.102:0/1956798137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 msgr2=0x7f82c4109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 --2- 172.21.15.102:0/1956798137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 0x7f82c4109bd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009ca0 tx=0x7f82bc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 -- 172.21.15.102:0/1956798137 shutdown_connections 2024-04-24T23:12:53.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 --2- 172.21.15.102:0/1956798137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 0x7f82c4109bd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 --2- 172.21.15.102:0/1956798137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4102980 0x7f82c4102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 --2- 172.21.15.102:0/1956798137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c4101790 0x7f82c4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 -- 172.21.15.102:0/1956798137 >> 172.21.15.102:0/1956798137 conn(0x7f82c40fcd20 msgr2=0x7f82c40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 -- 172.21.15.102:0/1956798137 shutdown_connections 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 -- 172.21.15.102:0/1956798137 wait complete. 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 Processor -- start 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 -- start start 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c4101790 0x7f82c4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4102980 0x7f82c4197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 0x7f82c419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410d3b0 con 0x7f82c4103340 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410d230 con 0x7f82c4102980 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.506+0000 7f82cb638640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410d530 con 0x7f82c4101790 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c9bd5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 0x7f82c419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c9bd5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 0x7f82c419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53586/0 (socket says 172.21.15.102:53586) 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c9bd5640 1 -- 172.21.15.102:0/525905510 learned_addr learned my addr 172.21.15.102:0/525905510 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4102980 0x7f82c4197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c93d4640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c4101790 0x7f82c4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c9bd5640 1 -- 172.21.15.102:0/525905510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c4101790 msgr2=0x7f82c4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c9bd5640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c4101790 0x7f82c4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c9bd5640 1 -- 172.21.15.102:0/525905510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4102980 msgr2=0x7f82c4197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c9bd5640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4102980 0x7f82c4197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c9bd5640 1 -- 172.21.15.102:0/525905510 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c419f6a0 con 0x7f82c4103340 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c93d4640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c4101790 0x7f82c4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82c9bd5640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 0x7f82c419f030 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009dc0 tx=0x7f82bc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82b67fc640 1 -- 172.21.15.102:0/525905510 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc004040 con 0x7f82c4103340 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82b67fc640 1 -- 172.21.15.102:0/525905510 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82bc004690 con 0x7f82c4103340 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82cb638640 1 -- 172.21.15.102:0/525905510 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c419f930 con 0x7f82c4103340 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82b67fc640 1 -- 172.21.15.102:0/525905510 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc02f4c0 con 0x7f82c4103340 2024-04-24T23:12:53.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.507+0000 7f82cb638640 1 -- 172.21.15.102:0/525905510 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c419ff00 con 0x7f82c4103340 2024-04-24T23:12:53.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82b67fc640 1 -- 172.21.15.102:0/525905510 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f82bc030650 con 0x7f82c4103340 2024-04-24T23:12:53.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82ce423740 1 Processor -- start 2024-04-24T23:12:53.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82ce423740 1 -- start start 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82ce423740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e54b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82ce423740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e54be30 0x55ed4e550930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82ce423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e550fa0 0x55ed4e553430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82ce423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e4aef00 con 0x55ed4e550fa0 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82ce423740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e457750 con 0x55ed4e54b440 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82ce423740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e4aaec0 con 0x55ed4e54be30 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c8bd3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e550fa0 0x55ed4e553430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c8bd3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e550fa0 0x55ed4e553430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53602/0 (socket says 172.21.15.102:53602) 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c8bd3640 1 -- 172.21.15.102:0/144513909 learned_addr learned my addr 172.21.15.102:0/144513909 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82b67fc640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82b00a2b90 0x7f82b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e54b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c93d4640 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e54be30 0x55ed4e550930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c8bd3640 1 -- 172.21.15.102:0/144513909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e54be30 msgr2=0x55ed4e550930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c9bd5640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82b00a2b90 0x7f82b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e54be30 0x55ed4e550930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c8bd3640 1 -- 172.21.15.102:0/144513909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 msgr2=0x55ed4e54b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e54b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c8bd3640 1 -- 172.21.15.102:0/144513909 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4e1c32d0 con 0x55ed4e550fa0 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.510+0000 7f82c93d4640 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e54be30 0x55ed4e550930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.511+0000 7f82b67fc640 1 -- 172.21.15.102:0/525905510 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f82bc0f0a70 con 0x7f82c4103340 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.511+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e550fa0 0x55ed4e553430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82c000add0 tx=0x7f82c000a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a5cc8ff99f338 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.511+0000 7f82af7fe640 1 -- 172.21.15.102:0/144513909 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c001b070 con 0x55ed4e550fa0 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.511+0000 7f82c9bd5640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82b00a2b90 0x7f82b00a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0bcfa0 tx=0x7f82bc0e7e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.511+0000 7f82af7fe640 1 -- 172.21.15.102:0/144513909 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82c000fd70 con 0x55ed4e550fa0 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.511+0000 7f82af7fe640 1 -- 172.21.15.102:0/144513909 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0014650 con 0x55ed4e550fa0 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.511+0000 7f82ce423740 1 -- 172.21.15.102:0/144513909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e550fa0 msgr2=0x55ed4e553430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.511+0000 7f82ce423740 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e550fa0 0x55ed4e553430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82c000add0 tx=0x7f82c000a490 comp rx=0 tx=0).stop 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 -- 172.21.15.102:0/144513909 shutdown_connections 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e550fa0 0x55ed4e553430 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e54be30 0x55ed4e550930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 --2- 172.21.15.102:0/144513909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e54b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 -- 172.21.15.102:0/144513909 >> 172.21.15.102:0/144513909 conn(0x55ed4e546ab0 msgr2=0x55ed4e546eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 -- 172.21.15.102:0/144513909 shutdown_connections 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 -- 172.21.15.102:0/144513909 wait complete. 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 -- start start 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 Processor -- start 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e5558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e555e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e55ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e553f50 con 0x55ed4e54be30 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e46acf0 con 0x55ed4e54b440 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e5540d0 con 0x55ed4e550fa0 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82ce423740 10 client.0 _ll_register_callbacks cb 0x55ed4e46a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:53.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82c93d4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e555e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82c93d4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e555e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53618/0 (socket says 172.21.15.102:53618) 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.512+0000 7f82c93d4640 1 -- 172.21.15.102:0/4233035614 learned_addr learned my addr 172.21.15.102:0/4233035614 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e55ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82c93d4640 1 -- 172.21.15.102:0/4233035614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 msgr2=0x55ed4e55ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82c93d4640 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e55ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82c93d4640 1 -- 172.21.15.102:0/4233035614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 msgr2=0x55ed4e5558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82c93d4640 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e5558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82c93d4640 1 -- 172.21.15.102:0/4233035614 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4e28d6b0 con 0x55ed4e54be30 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e55ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82c93d4640 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e555e10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f82b800d290 tx=0x7f82b800d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f2ef2d609f7ad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82acff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82acff9640 1 -- 172.21.15.102:0/4233035614 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8016070 con 0x55ed4e54be30 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82acff9640 1 -- 172.21.15.102:0/4233035614 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f82b8012870 con 0x55ed4e54be30 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82ce423740 1 -- 172.21.15.102:0/4233035614 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c000fd70 con 0x55ed4e54be30 2024-04-24T23:12:53.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82acff9640 1 -- 172.21.15.102:0/4233035614 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f82b8022050 con 0x55ed4e54be30 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82ce423740 10 client.15258 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82ce423740 1 -- 172.21.15.102:0/4233035614 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ed4e55fe70 con 0x55ed4e54be30 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82acff9640 1 -- 172.21.15.102:0/4233035614 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82c000fd70 con 0x55ed4e54be30 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82acff9640 1 -- 172.21.15.102:0/4233035614 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f82b8011ad0 con 0x55ed4e54be30 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82ce423740 10 client.15258 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82ce423740 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82ce423740 20 client.15258 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82ce423740 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed4e562510 0x55ed4e563980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82ce423740 4 client.15258 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.513+0000 7f82ce423740 1 -- 172.21.15.102:0/4233035614 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed4e4bc440 con 0x55ed4e562510 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.514+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed4e562510 0x55ed4e563980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.514+0000 7f82acff9640 10 client.15258 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:12:53.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.514+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed4e562510 0x55ed4e563980 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d9fbbe12d65c8e server_cookie=4f02586c89ee6c81 in_seq=0 out_seq=0 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82acff9640 1 -- 172.21.15.102:0/4233035614 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed4e4bc440 con 0x55ed4e562510 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82acff9640 10 client.15258 handle_command_reply: tid=0 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 client.15258 shutdown 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 -- 172.21.15.102:0/4233035614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 msgr2=0x55ed4e555e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e555e10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f82b800d290 tx=0x7f82b800d750 comp rx=0 tx=0).stop 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 -- 172.21.15.102:0/4233035614 shutdown_connections 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e55ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed4e562510 0x55ed4e563980 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e555e10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 --2- 172.21.15.102:0/4233035614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e5558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82acff9640 0 client.15258 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 -- 172.21.15.102:0/4233035614 >> 172.21.15.102:0/4233035614 conn(0x55ed4e546ab0 msgr2=0x55ed4e545ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 -- 172.21.15.102:0/4233035614 shutdown_connections 2024-04-24T23:12:53.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 1 -- 172.21.15.102:0/4233035614 wait complete. 2024-04-24T23:12:53.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.515+0000 7f82ce423740 20 client.15258 trim_cache size 0 max 16384 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82ce423740 1 Processor -- start 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82ce423740 1 -- start start 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82ce423740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e553e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82ce423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e55ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82ce423740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e541690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82ce423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e561cc0 con 0x55ed4e54be30 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82ce423740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e562390 con 0x55ed4e54b440 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82ce423740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e562a60 con 0x55ed4e550fa0 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82c93d4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e55ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82c93d4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e55ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53634/0 (socket says 172.21.15.102:53634) 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82c93d4640 1 -- 172.21.15.102:0/1733097825 learned_addr learned my addr 172.21.15.102:0/1733097825 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82c93d4640 1 -- 172.21.15.102:0/1733097825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 msgr2=0x55ed4e541690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82c93d4640 1 --2- 172.21.15.102:0/1733097825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e541690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82c93d4640 1 -- 172.21.15.102:0/1733097825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 msgr2=0x55ed4e553e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82c93d4640 1 --2- 172.21.15.102:0/1733097825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e553e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82c93d4640 1 -- 172.21.15.102:0/1733097825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4e54c210 con 0x55ed4e54be30 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.533+0000 7f82c93d4640 1 --2- 172.21.15.102:0/1733097825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e55ce10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f82b8000c00 tx=0x7f82b800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1144b3c9276c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82aeffd640 1 -- 172.21.15.102:0/1733097825 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b802b070 con 0x55ed4e54be30 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82aeffd640 1 -- 172.21.15.102:0/1733097825 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b801ada0 con 0x55ed4e54be30 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82aeffd640 1 -- 172.21.15.102:0/1733097825 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8016040 con 0x55ed4e54be30 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 -- 172.21.15.102:0/1733097825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 msgr2=0x55ed4e55ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 --2- 172.21.15.102:0/1733097825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e55ce10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f82b8000c00 tx=0x7f82b800b940 comp rx=0 tx=0).stop 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 -- 172.21.15.102:0/1733097825 shutdown_connections 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 --2- 172.21.15.102:0/1733097825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e541690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 --2- 172.21.15.102:0/1733097825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54be30 0x55ed4e55ce10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 --2- 172.21.15.102:0/1733097825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54b440 0x55ed4e553e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 -- 172.21.15.102:0/1733097825 >> 172.21.15.102:0/1733097825 conn(0x55ed4e546ab0 msgr2=0x55ed4e546e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 -- 172.21.15.102:0/1733097825 shutdown_connections 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 -- 172.21.15.102:0/1733097825 wait complete. 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.534+0000 7f82ce423740 1 -- start start 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82ce423740 1 Processor -- start 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82ce423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54b440 0x55ed4e558470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82ce423740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54be30 0x55ed4e5589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82ce423740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e55b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82ce423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e563800 con 0x55ed4e54b440 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82ce423740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e563130 con 0x55ed4e54be30 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82ce423740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed4e563ed0 con 0x55ed4e550fa0 2024-04-24T23:12:53.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82ce423740 10 client.0 _ll_register_callbacks cb 0x7f82b800bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c8bd3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54b440 0x55ed4e558470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c8bd3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54b440 0x55ed4e558470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53644/0 (socket says 172.21.15.102:53644) 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c8bd3640 1 -- 172.21.15.102:0/1881543143 learned_addr learned my addr 172.21.15.102:0/1881543143 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c93d4640 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54be30 0x55ed4e5589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c8bd3640 1 -- 172.21.15.102:0/1881543143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 msgr2=0x55ed4e55b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e55b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c8bd3640 1 -- 172.21.15.102:0/1881543143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54be30 msgr2=0x55ed4e5589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54be30 0x55ed4e5589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c8bd3640 1 -- 172.21.15.102:0/1881543143 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4e55bf70 con 0x55ed4e54b440 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c93d4640 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54be30 0x55ed4e5589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82c8bd3640 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54b440 0x55ed4e558470 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f82c000fa40 tx=0x7f82c000fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c26850d19400bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.535+0000 7f82acff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82acff9640 1 -- 172.21.15.102:0/1881543143 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c001b040 con 0x55ed4e54b440 2024-04-24T23:12:53.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82acff9640 1 -- 172.21.15.102:0/1881543143 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f82c00260b0 con 0x55ed4e54b440 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82ce423740 1 -- 172.21.15.102:0/1881543143 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82b801ada0 con 0x55ed4e54b440 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82acff9640 1 -- 172.21.15.102:0/1881543143 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f82c00262b0 con 0x55ed4e54b440 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82ce423740 10 client.15264 fetch_fsmap learned FSMap version 20 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82ce423740 1 -- 172.21.15.102:0/1881543143 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ed4e5c84c0 con 0x55ed4e54b440 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82acff9640 1 -- 172.21.15.102:0/1881543143 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b801ada0 con 0x55ed4e54b440 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82acff9640 1 -- 172.21.15.102:0/1881543143 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f82c0015040 con 0x55ed4e54b440 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82ce423740 10 client.15264 fetch_fsmap finished waiting for FSMap version 20 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82ce423740 10 client.15264 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82ce423740 20 client.15264 populate_metadata read hostname 'smithi102' 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82ce423740 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed4e5ccdb0 0x55ed4e5cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82ce423740 4 client.15264 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82ce423740 1 -- 172.21.15.102:0/1881543143 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "status"}) -- 0x55ed4e4b6790 con 0x55ed4e5ccdb0 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.536+0000 7f82c93d4640 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed4e5ccdb0 0x55ed4e5cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82acff9640 10 client.15264 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:12:53.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82c93d4640 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed4e5ccdb0 0x55ed4e5cbac0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ace1d119875aee3 server_cookie=2f17d607cf6aa9eb in_seq=0 out_seq=0 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82acff9640 1 -- 172.21.15.102:0/1881543143 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ed4e4b6790 con 0x55ed4e5ccdb0 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82acff9640 10 client.15264 handle_command_reply: tid=0 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 client.15264 shutdown 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 -- 172.21.15.102:0/1881543143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54b440 msgr2=0x55ed4e558470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54b440 0x55ed4e558470 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f82c000fa40 tx=0x7f82c000fd50 comp rx=0 tx=0).stop 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 -- 172.21.15.102:0/1881543143 shutdown_connections 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed4e550fa0 0x55ed4e55b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed4e54be30 0x55ed4e5589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed4e5ccdb0 0x55ed4e5cbac0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 --2- 172.21.15.102:0/1881543143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed4e54b440 0x55ed4e558470 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82acff9640 0 client.15264 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 -- 172.21.15.102:0/1881543143 >> 172.21.15.102:0/1881543143 conn(0x55ed4e546ab0 msgr2=0x55ed4e546e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 -- 172.21.15.102:0/1881543143 shutdown_connections 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 1 -- 172.21.15.102:0/1881543143 wait complete. 2024-04-24T23:12:53.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.537+0000 7f82ce423740 20 client.15264 trim_cache size 0 max 16384 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "97eecd66-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 0, 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 24497, 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 112.24622309199999, 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 20, 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 90, 2024-04-24T23:12:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T23:12:53.547 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 118.010223341 2024-04-24T23:12:53.547 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:12:53.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 -- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82b00a2b90 msgr2=0x7f82b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82b00a2b90 0x7f82b00a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0bcfa0 tx=0x7f82bc0e7e30 comp rx=0 tx=0).stop 2024-04-24T23:12:53.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 -- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 msgr2=0x7f82c419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 0x7f82c419f030 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009dc0 tx=0x7f82bc003ba0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 -- 172.21.15.102:0/525905510 shutdown_connections 2024-04-24T23:12:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f82b00a2b90 0x7f82b00a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4103340 0x7f82c419f030 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4102980 0x7f82c4197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 --2- 172.21.15.102:0/525905510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c4101790 0x7f82c4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 -- 172.21.15.102:0/525905510 >> 172.21.15.102:0/525905510 conn(0x7f82c40fcd20 msgr2=0x7f82c4105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 -- 172.21.15.102:0/525905510 shutdown_connections 2024-04-24T23:12:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.538+0000 7f82cb638640 1 -- 172.21.15.102:0/525905510 wait complete. 2024-04-24T23:12:53.550 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T23:12:53.560 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T23:12:53.560 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T23:12:53.560 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T23:12:53.560 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-24T23:12:53.561 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T23:12:53.561 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:12:53.561 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-24T23:12:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:53 smithi161 ceph-mon[32709]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T23:12:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:53 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1230034241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:53 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3797943037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:53 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1856164485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:53.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.656+0000 7f9b6949e640 1 -- 172.21.15.102:0/1041180930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 msgr2=0x7f9b64100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.656+0000 7f9b6949e640 1 --2- 172.21.15.102:0/1041180930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 0x7f9b64100ae0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009ca0 tx=0x7f9b5402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.656+0000 7f9b6949e640 1 -- 172.21.15.102:0/1041180930 shutdown_connections 2024-04-24T23:12:53.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.656+0000 7f9b6949e640 1 --2- 172.21.15.102:0/1041180930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64102270 0x7f9b64108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.656+0000 7f9b6949e640 1 --2- 172.21.15.102:0/1041180930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b641018b0 0x7f9b64101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.656+0000 7f9b6949e640 1 --2- 172.21.15.102:0/1041180930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 0x7f9b64100ae0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.656+0000 7f9b6949e640 1 -- 172.21.15.102:0/1041180930 >> 172.21.15.102:0/1041180930 conn(0x7f9b640fbc30 msgr2=0x7f9b640fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.656+0000 7f9b6949e640 1 -- 172.21.15.102:0/1041180930 shutdown_connections 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.656+0000 7f9b6949e640 1 -- 172.21.15.102:0/1041180930 wait complete. 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b6949e640 1 Processor -- start 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b6949e640 1 -- start start 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b6949e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 0x7f9b6406aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b6949e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b641018b0 0x7f9b6419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b6949e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64102270 0x7f9b6419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b6949e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6410c260 con 0x7f9b641006c0 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b6949e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6410c0e0 con 0x7f9b64102270 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b6949e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6410c3e0 con 0x7f9b641018b0 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b62ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 0x7f9b6406aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b62ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 0x7f9b6406aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53662/0 (socket says 172.21.15.102:53662) 2024-04-24T23:12:53.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.657+0000 7f9b62ffd640 1 -- 172.21.15.102:0/3264005742 learned_addr learned my addr 172.21.15.102:0/3264005742 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b627fc640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b641018b0 0x7f9b6419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b637fe640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64102270 0x7f9b6419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b62ffd640 1 -- 172.21.15.102:0/3264005742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b641018b0 msgr2=0x7f9b6419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b62ffd640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b641018b0 0x7f9b6419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b62ffd640 1 -- 172.21.15.102:0/3264005742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64102270 msgr2=0x7f9b6419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b62ffd640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64102270 0x7f9b6419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b62ffd640 1 -- 172.21.15.102:0/3264005742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6419f680 con 0x7f9b641006c0 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b62ffd640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 0x7f9b6406aa80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005e30 tx=0x7f9b54003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b4bfff640 1 -- 172.21.15.102:0/3264005742 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54032ec0 con 0x7f9b641006c0 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b4bfff640 1 -- 172.21.15.102:0/3264005742 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b540044f0 con 0x7f9b641006c0 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6419f8b0 con 0x7f9b641006c0 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b4bfff640 1 -- 172.21.15.102:0/3264005742 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54030d90 con 0x7f9b641006c0 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.658+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6419fe00 con 0x7f9b641006c0 2024-04-24T23:12:53.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.659+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6404f500 con 0x7f9b641006c0 2024-04-24T23:12:53.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.660+0000 7f9b4bfff640 1 -- 172.21.15.102:0/3264005742 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9b540305c0 con 0x7f9b641006c0 2024-04-24T23:12:53.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.660+0000 7f9b4bfff640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f9b4c0a2dd0 0x7f9b4c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.660+0000 7f9b627fc640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f9b4c0a2dd0 0x7f9b4c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.661+0000 7f9b4bfff640 1 -- 172.21.15.102:0/3264005742 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f9b540f07a0 con 0x7f9b641006c0 2024-04-24T23:12:53.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.661+0000 7f9b627fc640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f9b4c0a2dd0 0x7f9b4c0a5290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005fd0 tx=0x7f9b58005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.665+0000 7f9b4bfff640 1 -- 172.21.15.102:0/3264005742 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9b540bd6d0 con 0x7f9b641006c0 2024-04-24T23:12:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:53 smithi183 ceph-mon[33137]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T23:12:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:53 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1230034241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:53 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3797943037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:53 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1856164485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:53.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:53.888+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f9b64062340 con 0x7f9b641006c0 2024-04-24T23:12:54.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.335+0000 7f9b4bfff640 1 -- 172.21.15.102:0/3264005742 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f9b540bcfd0 con 0x7f9b641006c0 2024-04-24T23:12:54.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f9b4c0a2dd0 msgr2=0x7f9b4c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f9b4c0a2dd0 0x7f9b4c0a5290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005fd0 tx=0x7f9b58005850 comp rx=0 tx=0).stop 2024-04-24T23:12:54.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 msgr2=0x7f9b6406aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 0x7f9b6406aa80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005e30 tx=0x7f9b54003ba0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 shutdown_connections 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f9b4c0a2dd0 0x7f9b4c0a5290 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b64102270 0x7f9b6419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b641018b0 0x7f9b6419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 --2- 172.21.15.102:0/3264005742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b641006c0 0x7f9b6406aa80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 >> 172.21.15.102:0/3264005742 conn(0x7f9b640fbc30 msgr2=0x7f9b64104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 shutdown_connections 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.337+0000 7f9b6949e640 1 -- 172.21.15.102:0/3264005742 wait complete. 2024-04-24T23:12:54.358 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-24T23:12:54.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.462+0000 7fa3b69a2640 1 -- 172.21.15.102:0/1572389273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b0102250 msgr2=0x7fa3b0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.462+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/1572389273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b0102250 0x7fa3b0108ae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009ca0 tx=0x7fa3a802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.462+0000 7fa3b69a2640 1 -- 172.21.15.102:0/1572389273 shutdown_connections 2024-04-24T23:12:54.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.462+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/1572389273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b0102250 0x7fa3b0108ae0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.462+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/1572389273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b0101890 0x7fa3b0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.462+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/1572389273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b01006a0 0x7fa3b0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.462+0000 7fa3b69a2640 1 -- 172.21.15.102:0/1572389273 >> 172.21.15.102:0/1572389273 conn(0x7fa3b00fbc10 msgr2=0x7fa3b00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.462+0000 7fa3b69a2640 1 -- 172.21.15.102:0/1572389273 shutdown_connections 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.462+0000 7fa3b69a2640 1 -- 172.21.15.102:0/1572389273 wait complete. 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3b69a2640 1 Processor -- start 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3b69a2640 1 -- start start 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3b69a2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b01006a0 0x7fa3b0196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3b69a2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b0101890 0x7fa3b0196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3b69a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b0102250 0x7fa3b019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3b69a2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b010c1d0 con 0x7fa3b01006a0 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3b69a2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b010c050 con 0x7fa3b0101890 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3b69a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b010c350 con 0x7fa3b0102250 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3affff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b01006a0 0x7fa3b0196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3affff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b01006a0 0x7fa3b0196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53706/0 (socket says 172.21.15.102:53706) 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3affff640 1 -- 172.21.15.102:0/260340735 learned_addr learned my addr 172.21.15.102:0/260340735 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3b4f3f640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b0102250 0x7fa3b019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3af7fe640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b0101890 0x7fa3b0196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3affff640 1 -- 172.21.15.102:0/260340735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b0102250 msgr2=0x7fa3b019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3affff640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b0102250 0x7fa3b019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3affff640 1 -- 172.21.15.102:0/260340735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b0101890 msgr2=0x7fa3b0196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3affff640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b0101890 0x7fa3b0196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.463+0000 7fa3affff640 1 -- 172.21.15.102:0/260340735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b019e430 con 0x7fa3b01006a0 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.464+0000 7fa3affff640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b01006a0 0x7fa3b0196810 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000d770 tx=0x7fa3a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.464+0000 7fa3ad7fa640 1 -- 172.21.15.102:0/260340735 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0017070 con 0x7fa3b01006a0 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.464+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b019e720 con 0x7fa3b01006a0 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.464+0000 7fa3ad7fa640 1 -- 172.21.15.102:0/260340735 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3a0009c30 con 0x7fa3b01006a0 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.464+0000 7fa3ad7fa640 1 -- 172.21.15.102:0/260340735 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0003d00 con 0x7fa3b01006a0 2024-04-24T23:12:54.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.464+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b019ecc0 con 0x7fa3b01006a0 2024-04-24T23:12:54.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.465+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b01079c0 con 0x7fa3b01006a0 2024-04-24T23:12:54.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.466+0000 7fa3ad7fa640 1 -- 172.21.15.102:0/260340735 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7fa3a0003ea0 con 0x7fa3b01006a0 2024-04-24T23:12:54.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.467+0000 7fa3ad7fa640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa3980a2c80 0x7fa3980a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.467+0000 7fa3af7fe640 1 -- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa3980a2c80 msgr2=0x7fa3980a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3611055373 2024-04-24T23:12:54.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.467+0000 7fa3ad7fa640 1 -- 172.21.15.102:0/260340735 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa3a00ced40 con 0x7fa3b01006a0 2024-04-24T23:12:54.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.467+0000 7fa3af7fe640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa3980a2c80 0x7fa3980a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:12:54.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.471+0000 7fa3ad7fa640 1 -- 172.21.15.102:0/260340735 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa3a0009de0 con 0x7fa3b01006a0 2024-04-24T23:12:54.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:54 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3264005742' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T23:12:54.609 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:54 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: ignoring --setuser ceph since I am not root 2024-04-24T23:12:54.610 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:54 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: ignoring --setgroup ceph since I am not root 2024-04-24T23:12:54.610 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:54 smithi161 ceph-mgr[34110]: -- 172.21.15.161:0/2461791071 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c771f65a0 con 0x556c771cb000 2024-04-24T23:12:54.610 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:54 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:54.518+0000 7f6892885200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:12:54.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.655+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fa3b002d170 con 0x7fa3b01006a0 2024-04-24T23:12:54.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.659+0000 7fa3ad7fa640 1 -- 172.21.15.102:0/260340735 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fa3b002d170 con 0x7fa3b01006a0 2024-04-24T23:12:54.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa3980a2c80 msgr2=0x7fa3980a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa3980a2c80 0x7fa3980a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b01006a0 msgr2=0x7fa3b0196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b01006a0 0x7fa3b0196810 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000d770 tx=0x7fa3a000dc30 comp rx=0 tx=0).stop 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 shutdown_connections 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7fa3980a2c80 0x7fa3980a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b0102250 0x7fa3b019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b0101890 0x7fa3b0196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 --2- 172.21.15.102:0/260340735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b01006a0 0x7fa3b0196810 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.661+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 >> 172.21.15.102:0/260340735 conn(0x7fa3b00fbc10 msgr2=0x7fa3b0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.662+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 shutdown_connections 2024-04-24T23:12:54.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.662+0000 7fa3b69a2640 1 -- 172.21.15.102:0/260340735 wait complete. 2024-04-24T23:12:54.673 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:54 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3264005742' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T23:12:54.674 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:54 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: ignoring --setuser ceph since I am not root 2024-04-24T23:12:54.674 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:54 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: ignoring --setgroup ceph since I am not root 2024-04-24T23:12:54.674 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:54 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:54.521+0000 7f8bf876d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:12:54.675 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-24T23:12:54.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:54 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3264005742' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T23:12:54.698 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:54 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: ignoring --setuser ceph since I am not root 2024-04-24T23:12:54.698 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:54 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: ignoring --setgroup ceph since I am not root 2024-04-24T23:12:54.699 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:54 smithi183 ceph-mgr[35005]: -- 172.21.15.183:0/4158857700 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e4097a5a0 con 0x563e4094f000 2024-04-24T23:12:54.699 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:54 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:54.513+0000 7f55f5e8c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:12:54.699 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:54 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:54.695+0000 7f55f5e8c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:12:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.774+0000 7f780aa2c640 1 -- 172.21.15.102:0/3348130714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804102250 msgr2=0x7f7804108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.774+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3348130714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804102250 0x7f7804108ae0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009ca0 tx=0x7f77fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.774+0000 7f780aa2c640 1 -- 172.21.15.102:0/3348130714 shutdown_connections 2024-04-24T23:12:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.774+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3348130714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804102250 0x7f7804108ae0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.774+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3348130714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7804101890 0x7f7804101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.774+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3348130714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78041006a0 0x7f7804100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.774+0000 7f780aa2c640 1 -- 172.21.15.102:0/3348130714 >> 172.21.15.102:0/3348130714 conn(0x7f78040fbc10 msgr2=0x7f78040fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.774+0000 7f780aa2c640 1 -- 172.21.15.102:0/3348130714 shutdown_connections 2024-04-24T23:12:54.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.774+0000 7f780aa2c640 1 -- 172.21.15.102:0/3348130714 wait complete. 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f780aa2c640 1 Processor -- start 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f780aa2c640 1 -- start start 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f780aa2c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78041006a0 0x7f78041045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f780aa2c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804101890 0x7f7804102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f780aa2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7804102250 0x7f78041031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f780aa2c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f780410c1d0 con 0x7f7804101890 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f780aa2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f780410c050 con 0x7f7804102250 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f780aa2c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f780410c350 con 0x7f78041006a0 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f78037fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804101890 0x7f7804102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f78037fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804101890 0x7f7804102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53728/0 (socket says 172.21.15.102:53728) 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f7803fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78041006a0 0x7f78041045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f78037fe640 1 -- 172.21.15.102:0/3417501411 learned_addr learned my addr 172.21.15.102:0/3417501411 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.775+0000 7f7808fc9640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7804102250 0x7f78041031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f78037fe640 1 -- 172.21.15.102:0/3417501411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78041006a0 msgr2=0x7f78041045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f78037fe640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78041006a0 0x7f78041045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f78037fe640 1 -- 172.21.15.102:0/3417501411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7804102250 msgr2=0x7f78041031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f78037fe640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7804102250 0x7f78041031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f78037fe640 1 -- 172.21.15.102:0/3417501411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7804103a70 con 0x7f7804101890 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f7803fff640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78041006a0 0x7f78041045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f78037fe640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804101890 0x7f7804102c70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f77f800e9f0 tx=0x7f77f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f78017fa640 1 -- 172.21.15.102:0/3417501411 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f800cd80 con 0x7f7804101890 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f78017fa640 1 -- 172.21.15.102:0/3417501411 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77f80108e0 con 0x7f7804101890 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78041a1a40 con 0x7f7804101890 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f78017fa640 1 -- 172.21.15.102:0/3417501411 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8018920 con 0x7f7804101890 2024-04-24T23:12:54.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.776+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78041a1f60 con 0x7f7804101890 2024-04-24T23:12:54.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.777+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78041a2190 con 0x7f7804101890 2024-04-24T23:12:54.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.778+0000 7f78017fa640 1 -- 172.21.15.102:0/3417501411 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f77f8010a90 con 0x7f7804101890 2024-04-24T23:12:54.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.779+0000 7f78017fa640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f77ec0a2c80 0x7f77ec0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.779+0000 7f7803fff640 1 -- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f77ec0a2c80 msgr2=0x7f77ec0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3611055373 2024-04-24T23:12:54.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.779+0000 7f78017fa640 1 -- 172.21.15.102:0/3417501411 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f77f80cf3a0 con 0x7f7804101890 2024-04-24T23:12:54.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.779+0000 7f7803fff640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f77ec0a2c80 0x7f77ec0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:12:54.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.783+0000 7f78017fa640 1 -- 172.21.15.102:0/3417501411 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f77f809bc80 con 0x7f7804101890 2024-04-24T23:12:54.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.980+0000 7f7803fff640 1 -- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f77ec0a2c80 msgr2=0x7f77ec0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3611055373 2024-04-24T23:12:54.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:54.980+0000 7f7803fff640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f77ec0a2c80 0x7f77ec0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:12:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.006+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f78041a2520 con 0x7f7804101890 2024-04-24T23:12:55.012 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:54 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:54.798+0000 7f55f5e8c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:12:55.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.010+0000 7f78017fa640 1 -- 172.21.15.102:0/3417501411 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f77f809ff80 con 0x7f7804101890 2024-04-24T23:12:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f77ec0a2c80 msgr2=0x7f77ec0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:12:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f77ec0a2c80 0x7f77ec0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804101890 msgr2=0x7f7804102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804101890 0x7f7804102c70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f77f800e9f0 tx=0x7f77f800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 shutdown_connections 2024-04-24T23:12:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f77ec0a2c80 0x7f77ec0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7804102250 0x7f78041031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7804101890 0x7f7804102c70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 --2- 172.21.15.102:0/3417501411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78041006a0 0x7f78041045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 >> 172.21.15.102:0/3417501411 conn(0x7f78040fbc10 msgr2=0x7f78040fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:55.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 shutdown_connections 2024-04-24T23:12:55.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:55.013+0000 7f780aa2c640 1 -- 172.21.15.102:0/3417501411 wait complete. 2024-04-24T23:12:55.024 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:54 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:54.713+0000 7f8bf876d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:12:55.025 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:54 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:54.848+0000 7f8bf876d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:12:55.025 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-24T23:12:55.051 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-24T23:12:55.051 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-24T23:12:55.051 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-24T23:12:55.051 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:12:55.051 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:55.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:55.052 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-04-24T23:12:55.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:55.067 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link delete ceph-brx 2024-04-24T23:12:55.109 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:54 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:54.704+0000 7f6892885200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:12:55.109 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:54 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:54.806+0000 7f6892885200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:12:55.133 INFO:teuthology.orchestra.run.smithi102.stderr:Cannot find device "ceph-brx" 2024-04-24T23:12:55.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:12:55.135 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-24T23:12:55.135 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T23:12:55.322 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:55 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:55.177+0000 7f8bf876d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:12:55.322 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:55 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:55.267+0000 7f8bf876d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:12:55.418 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:55.512 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:55 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:55.105+0000 7f55f5e8c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:12:55.512 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:55 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:55.189+0000 7f55f5e8c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:12:55.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:55 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3264005742' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T23:12:55.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:55 smithi183 ceph-mon[33137]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-24T23:12:55.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:55 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/260340735' entity='client.admin' 2024-04-24T23:12:55.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:55 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3417501411' entity='client.admin' 2024-04-24T23:12:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:55 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3264005742' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T23:12:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:55 smithi161 ceph-mon[32709]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-24T23:12:55.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:55 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/260340735' entity='client.admin' 2024-04-24T23:12:55.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:55 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3417501411' entity='client.admin' 2024-04-24T23:12:55.610 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:55 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:55.117+0000 7f6892885200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:12:55.610 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:55 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:55.202+0000 7f6892885200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:12:55.681 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:55 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3264005742' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T23:12:55.681 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:55 smithi102 ceph-mon[29216]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-24T23:12:55.682 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:55 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/260340735' entity='client.admin' 2024-04-24T23:12:55.682 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:55 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3417501411' entity='client.admin' 2024-04-24T23:12:56.712 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:56 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:56.369+0000 7f55f5e8c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:12:56.713 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:56 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:56.539+0000 7f55f5e8c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:12:56.713 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:56 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:56.710+0000 7f55f5e8c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:12:56.728 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:56 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:56.387+0000 7f6892885200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:12:56.729 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:56 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:56.557+0000 7f6892885200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:12:56.821 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:56 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:56.487+0000 7f8bf876d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:12:56.822 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:56 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:56.660+0000 7f8bf876d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:12:57.015 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:56 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:56.727+0000 7f6892885200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:12:57.103 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:56 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:56.836+0000 7f8bf876d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:12:57.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.206+0000 7f039db78700 1 -- 172.21.15.102:0/3730865500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806a7d0 msgr2=0x7f039806ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.206+0000 7f039db78700 1 --2- 172.21.15.102:0/3730865500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806a7d0 0x7f039806ac30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f038c009990 tx=0x7f038c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.206+0000 7f039db78700 1 -- 172.21.15.102:0/3730865500 shutdown_connections 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.206+0000 7f039db78700 1 --2- 172.21.15.102:0/3730865500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f039806b170 0x7f039810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.206+0000 7f039db78700 1 --2- 172.21.15.102:0/3730865500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03981082a0 0x7f0398108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.206+0000 7f039db78700 1 --2- 172.21.15.102:0/3730865500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806a7d0 0x7f039806ac30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.206+0000 7f039db78700 1 -- 172.21.15.102:0/3730865500 >> 172.21.15.102:0/3730865500 conn(0x7f03980780a0 msgr2=0x7f03980784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.206+0000 7f039db78700 1 -- 172.21.15.102:0/3730865500 shutdown_connections 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.206+0000 7f039db78700 1 -- 172.21.15.102:0/3730865500 wait complete. 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f039db78700 1 Processor -- start 2024-04-24T23:12:57.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f039db78700 1 -- start start 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f039db78700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f039806a7d0 0x7f039819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f039db78700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806b170 0x7f039819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f039db78700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03981082a0 0x7f0398198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f039db78700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f03981123d0 con 0x7f039806b170 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f039db78700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0398112250 con 0x7f039806a7d0 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f039db78700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0398112550 con 0x7f03981082a0 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f0396ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806b170 0x7f039819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f0397fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03981082a0 0x7f0398198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f03977fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f039806a7d0 0x7f039819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f0396ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806b170 0x7f039819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37666/0 (socket says 172.21.15.102:37666) 2024-04-24T23:12:57.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.207+0000 7f0396ffd700 1 -- 172.21.15.102:0/1846307464 learned_addr learned my addr 172.21.15.102:0/1846307464 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0396ffd700 1 -- 172.21.15.102:0/1846307464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03981082a0 msgr2=0x7f0398198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0396ffd700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03981082a0 0x7f0398198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0396ffd700 1 -- 172.21.15.102:0/1846307464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f039806a7d0 msgr2=0x7f039819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0396ffd700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f039806a7d0 0x7f039819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0396ffd700 1 -- 172.21.15.102:0/1846307464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0398199050 con 0x7f039806b170 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0397fff700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03981082a0 0x7f0398198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f03977fe700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f039806a7d0 0x7f039819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0396ffd700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806b170 0x7f039819ebe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f038c000c00 tx=0x7f038c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0394ff9700 1 -- 172.21.15.102:0/1846307464 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c027070 con 0x7f039806b170 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03981992e0 con 0x7f039806b170 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0394ff9700 1 -- 172.21.15.102:0/1846307464 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f038c022470 con 0x7f039806b170 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f0394ff9700 1 -- 172.21.15.102:0/1846307464 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c00b610 con 0x7f039806b170 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.208+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03981012f0 con 0x7f039806b170 2024-04-24T23:12:57.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.209+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0398199640 con 0x7f039806b170 2024-04-24T23:12:57.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.211+0000 7f0394ff9700 1 -- 172.21.15.102:0/1846307464 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f038c02a070 con 0x7f039806b170 2024-04-24T23:12:57.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.212+0000 7f0394ff9700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f03800a2cf0 0x7f03800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:57.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.212+0000 7f03977fe700 1 -- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f03800a2cf0 msgr2=0x7f03800a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3611055373 2024-04-24T23:12:57.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.212+0000 7f0394ff9700 1 -- 172.21.15.102:0/1846307464 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f038c0d00e0 con 0x7f039806b170 2024-04-24T23:12:57.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.212+0000 7f03977fe700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f03800a2cf0 0x7f03800a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:12:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.215+0000 7f0394ff9700 1 -- 172.21.15.102:0/1846307464 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f038c013070 con 0x7f039806b170 2024-04-24T23:12:57.262 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:56 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:56.997+0000 7f55f5e8c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:12:57.262 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:57 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:57.090+0000 7f55f5e8c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:12:57.311 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:57 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:57.014+0000 7f6892885200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:12:57.311 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:57 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:57.107+0000 7f6892885200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:12:57.384 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:57 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:57.149+0000 7f8bf876d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:12:57.384 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:57 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:57.250+0000 7f8bf876d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:12:57.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.413+0000 7f03977fe700 1 -- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f03800a2cf0 msgr2=0x7f03800a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3611055373 2024-04-24T23:12:57.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.413+0000 7f03977fe700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f03800a2cf0 0x7f03800a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:12:57.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.471+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0398068f20 con 0x7f039806b170 2024-04-24T23:12:57.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.471+0000 7f0394ff9700 1 -- 172.21.15.102:0/1846307464 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f038c09d0a0 con 0x7f039806b170 2024-04-24T23:12:57.473 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:12:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f03800a2cf0 msgr2=0x7f03800a51b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:12:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f03800a2cf0 0x7f03800a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806b170 msgr2=0x7f039819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806b170 0x7f039819ebe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f038c000c00 tx=0x7f038c004790 comp rx=0 tx=0).stop 2024-04-24T23:12:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 shutdown_connections 2024-04-24T23:12:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03981082a0 0x7f0398198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f039806a7d0 0x7f039819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f03800a2cf0 0x7f03800a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 --2- 172.21.15.102:0/1846307464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f039806b170 0x7f039819ebe0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.473+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 >> 172.21.15.102:0/1846307464 conn(0x7f03980780a0 msgr2=0x7f03980ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.474+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 shutdown_connections 2024-04-24T23:12:57.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:57.474+0000 7f039db78700 1 -- 172.21.15.102:0/1846307464 wait complete. 2024-04-24T23:12:57.524 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:57 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:57.294+0000 7f55f5e8c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:12:57.525 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:57 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:57.485+0000 7f55f5e8c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:12:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:12:57 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1846307464' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:12:57.609 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:57 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:57.310+0000 7f6892885200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:12:57.610 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:57 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:57.502+0000 7f6892885200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:12:57.687 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:12:57 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1846307464' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:12:57.687 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:57 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:57.461+0000 7f8bf876d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:12:58.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:12:57 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1846307464' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:12:58.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:57 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:57.682+0000 7f8bf876d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:12:58.260 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:12:58.260 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:12:58.260 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi102.front.sepia.ceph.com 2024-04-24T23:12:58.260 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:12:58.260 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:58.260 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:12:58.260 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:12:58.260 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:12:58.260 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:12:58.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:58.261 DEBUG:teuthology.orchestra.run.smithi102:> ip addr 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 ::1/128 scope host 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dc brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f0 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:12:58.278 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dd brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:58.279 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f1 2024-04-24T23:12:58.279 INFO:teuthology.orchestra.run.smithi102.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:12:58.279 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:68 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:58.279 INFO:teuthology.orchestra.run.smithi102.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:12:58.279 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:69 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:58.279 INFO:teuthology.orchestra.run.smithi102.stdout: inet 172.21.15.102/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:12:58.279 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft 42050sec preferred_lft 42050sec 2024-04-24T23:12:58.279 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::cd07:e13c:958c:c87a/64 scope link noprefixroute 2024-04-24T23:12:58.279 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:12:58.279 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:12:58.280 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:58.280 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-24T23:12:58.280 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:12:58.280 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr flush dev ceph-brx 2024-04-24T23:12:58.280 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set ceph-brx up 2024-04-24T23:12:58.280 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:12:58.280 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-24T23:12:58.359 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:58 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:58.106+0000 7f6892885200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:12:58.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:58.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:58.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:58.466 DEBUG:teuthology.orchestra.run.smithi102:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:12:58.478 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:58 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:58.311+0000 7f8bf876d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:12:58.492 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-04-24T23:12:58.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:58.494 DEBUG:teuthology.orchestra.run.smithi102:> route 2024-04-24T23:12:58.505 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:58 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:58.085+0000 7f55f5e8c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:12:58.559 INFO:teuthology.orchestra.run.smithi102.stdout:Kernel IP routing table 2024-04-24T23:12:58.560 INFO:teuthology.orchestra.run.smithi102.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:12:58.560 INFO:teuthology.orchestra.run.smithi102.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:12:58.560 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:12:58.560 INFO:teuthology.orchestra.run.smithi102.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:12:58.560 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:58.560 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-24T23:12:58.560 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:12:58.560 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:12:58.561 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:12:58.561 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-24T23:12:58.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:58.762 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:58 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:58.503+0000 7f55f5e8c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:12:58.762 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:58 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:58.600+0000 7f55f5e8c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:12:58.824 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:58 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:58.526+0000 7f6892885200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:12:58.824 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:58 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:58.623+0000 7f6892885200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:12:58.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:58.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:58.834 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-04-24T23:12:58.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:58.849 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list-id 2024-04-24T23:12:58.907 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:58.907 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-24T23:12:58.907 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:12:58.908 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T23:12:58.908 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-24T23:12:58.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:59.003 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:58 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:58.776+0000 7f8bf876d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:12:59.003 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:58 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:58.879+0000 7f8bf876d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:12:59.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:59.024 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T23:12:59.024 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:59.025 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-24T23:12:59.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T23:12:59.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:12:59.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T23:12:59.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T23:12:59.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T23:12:59.025 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-24T23:12:59.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:59.110 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:58 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:58.823+0000 7f6892885200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:12:59.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:59.150 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:59.151 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-04-24T23:12:59.151 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set brx.0 up 2024-04-24T23:12:59.151 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:12:59.151 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-04-24T23:12:59.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:59.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:12:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:59.228 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-24T23:12:59.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:12:59.228 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:59.244 INFO:teuthology.orchestra.run.smithi102.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:12:59.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:12:59.244 DEBUG:teuthology.orchestra.run.smithi102:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:12:59.262 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:58 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:58.800+0000 7f55f5e8c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:12:59.299 DEBUG:teuthology.orchestra.run.smithi102:> sudo modprobe fuse 2024-04-24T23:12:59.322 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:59 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:59.094+0000 7f8bf876d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:12:59.346 DEBUG:teuthology.orchestra.run.smithi102:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:12:59.402 INFO:teuthology.orchestra.run.smithi102.stdout:/proc 2024-04-24T23:12:59.402 INFO:teuthology.orchestra.run.smithi102.stdout:/sys 2024-04-24T23:12:59.402 INFO:teuthology.orchestra.run.smithi102.stdout:/dev 2024-04-24T23:12:59.402 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/security 2024-04-24T23:12:59.402 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/shm 2024-04-24T23:12:59.402 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/pts 2024-04-24T23:12:59.402 INFO:teuthology.orchestra.run.smithi102.stdout:/run 2024-04-24T23:12:59.402 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/cgroup 2024-04-24T23:12:59.402 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/pstore 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/bpf 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/ 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/selinux 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/hugepages 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/mqueue 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/debug 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/tracing 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/config 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/fuse/connections 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:12:59.403 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/1000 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/b37af7bc618d194b3d157a7745df4b7cd246bf3b2997b3b63afb34b36abe2a52/merged 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/5cb6d3be9ec6b13a1c5e4f5900d50f640ab8fad7bd11d680597e061b9130061b/merged 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/0 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/a88b806663148344deab9cd444a30f0a2d1ad565806c8e3e4114d7e0b3856f6f/merged 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/7c7a91239e1e6cfd04e10cb20e0963ff56edc7bedc67f83f8f01d79cb45fa217/merged 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/6356abbb4354ff39a1319954b08c05b808f96767025adec97936d9690a4ae063/merged 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/d8efc88cf85bfcd5283ac48fe6b60c70e5282c0a356c3448759d2ff1aac395fe/merged 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/a3ab49f1adb05376f109064510c69f05b933a24176851f7bee6e7678ac7c2e28/merged 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/e19919082e2f1728915ce570061e1c7c4dd62d2765827d45bf578b1c3265d7bb/merged 2024-04-24T23:12:59.404 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/95373575959a4e3fbb58cda7e913bd42d6b9d6e421568fb414044b2062538389/merged 2024-04-24T23:12:59.405 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/8768df6318693e8df0beb7e78cb18abd090d5e16076ec67337206fa1cec9077a/merged 2024-04-24T23:12:59.405 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns 2024-04-24T23:12:59.405 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:12:59.405 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:12:59.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:59.405 DEBUG:teuthology.orchestra.run.smithi102:> ls /sys/fs/fuse/connections 2024-04-24T23:12:59.457 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T23:12:59.457 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/77976509-2cd8-41de-8489-e8b8dd738f18 --client_fs=cephfs) 2024-04-24T23:12:59.500 DEBUG:teuthology.orchestra.run.smithi102:> sudo modprobe fuse 2024-04-24T23:12:59.523 DEBUG:teuthology.orchestra.run.smithi102:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:12:59.580 INFO:teuthology.orchestra.run.smithi102.stdout:/proc 2024-04-24T23:12:59.580 INFO:teuthology.orchestra.run.smithi102.stdout:/sys 2024-04-24T23:12:59.580 INFO:teuthology.orchestra.run.smithi102.stdout:/dev 2024-04-24T23:12:59.580 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/security 2024-04-24T23:12:59.580 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/shm 2024-04-24T23:12:59.580 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/pts 2024-04-24T23:12:59.580 INFO:teuthology.orchestra.run.smithi102.stdout:/run 2024-04-24T23:12:59.580 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/cgroup 2024-04-24T23:12:59.580 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/pstore 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/bpf 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/ 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/selinux 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/hugepages 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/mqueue 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/debug 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/tracing 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/config 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/fuse/connections 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:12:59.581 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/1000 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/b37af7bc618d194b3d157a7745df4b7cd246bf3b2997b3b63afb34b36abe2a52/merged 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/5cb6d3be9ec6b13a1c5e4f5900d50f640ab8fad7bd11d680597e061b9130061b/merged 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/0 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/a88b806663148344deab9cd444a30f0a2d1ad565806c8e3e4114d7e0b3856f6f/merged 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/7c7a91239e1e6cfd04e10cb20e0963ff56edc7bedc67f83f8f01d79cb45fa217/merged 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/6356abbb4354ff39a1319954b08c05b808f96767025adec97936d9690a4ae063/merged 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/d8efc88cf85bfcd5283ac48fe6b60c70e5282c0a356c3448759d2ff1aac395fe/merged 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/a3ab49f1adb05376f109064510c69f05b933a24176851f7bee6e7678ac7c2e28/merged 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/e19919082e2f1728915ce570061e1c7c4dd62d2765827d45bf578b1c3265d7bb/merged 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/95373575959a4e3fbb58cda7e913bd42d6b9d6e421568fb414044b2062538389/merged 2024-04-24T23:12:59.582 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/8768df6318693e8df0beb7e78cb18abd090d5e16076ec67337206fa1cec9077a/merged 2024-04-24T23:12:59.583 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns 2024-04-24T23:12:59.583 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:12:59.583 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:12:59.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:59.583 DEBUG:teuthology.orchestra.run.smithi102:> ls /sys/fs/fuse/connections 2024-04-24T23:12:59.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:2024-04-24T23:12:59.598+0000 7fd112322480 -1 init, newargv = 0x55570d714190 newargc=15 2024-04-24T23:12:59.600 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:ceph-fuse[81764]: starting ceph client 2024-04-24T23:12:59.609 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:59 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:59.352+0000 7f6892885200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:12:59.609 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:59 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:59.444+0000 7f6892885200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:12:59.614 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:ceph-fuse[81764]: starting fuse 2024-04-24T23:12:59.655 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:59 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:59.321+0000 7f55f5e8c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:12:59.655 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:59 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:59.411+0000 7f55f5e8c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:12:59.888 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:59 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:59.691+0000 7f6892885200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:12:59.888 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:59 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:59.829+0000 7f6892885200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:12:59.993 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:59 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:59.650+0000 7f8bf876d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:12:59.993 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:59 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:59.743+0000 7f8bf876d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:12:59.998 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:59 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:59.652+0000 7f55f5e8c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:12:59.998 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:59 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:59.785+0000 7f55f5e8c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:12:59.998 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:59 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:59.880+0000 7f55f5e8c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:12:59.998 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:12:59 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:12:59.996+0000 7f55f5e8c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:13:00.149 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:12:59 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:12:59.928+0000 7f6892885200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:13:00.149 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:13:00 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:13:00.052+0000 7f6892885200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:13:00.262 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:13:00 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:13:00.089+0000 7f55f5e8c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:13:00.322 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:12:59 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:12:59.991+0000 7f8bf876d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:13:00.322 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:00 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:00.126+0000 7f8bf876d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:13:00.322 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:00 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:00.221+0000 7f8bf876d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:13:00.609 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:13:00 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:13:00.148+0000 7f6892885200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:13:00.715 DEBUG:teuthology.orchestra.run.smithi102:> sudo modprobe fuse 2024-04-24T23:13:00.737 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:00 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:00.342+0000 7f8bf876d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:13:00.737 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:00 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:00.437+0000 7f8bf876d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:13:00.741 DEBUG:teuthology.orchestra.run.smithi102:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:13:00.797 INFO:teuthology.orchestra.run.smithi102.stdout:/proc 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/sys 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/dev 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/security 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/shm 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/pts 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/run 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/cgroup 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/pstore 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/bpf 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/ 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/selinux 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:13:00.798 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/hugepages 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/mqueue 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/debug 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/tracing 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/config 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/fuse/connections 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/1000 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:13:00.799 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/b37af7bc618d194b3d157a7745df4b7cd246bf3b2997b3b63afb34b36abe2a52/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/5cb6d3be9ec6b13a1c5e4f5900d50f640ab8fad7bd11d680597e061b9130061b/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/0 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/a88b806663148344deab9cd444a30f0a2d1ad565806c8e3e4114d7e0b3856f6f/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/7c7a91239e1e6cfd04e10cb20e0963ff56edc7bedc67f83f8f01d79cb45fa217/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/6356abbb4354ff39a1319954b08c05b808f96767025adec97936d9690a4ae063/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/d8efc88cf85bfcd5283ac48fe6b60c70e5282c0a356c3448759d2ff1aac395fe/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/a3ab49f1adb05376f109064510c69f05b933a24176851f7bee6e7678ac7c2e28/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/e19919082e2f1728915ce570061e1c7c4dd62d2765827d45bf578b1c3265d7bb/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/95373575959a4e3fbb58cda7e913bd42d6b9d6e421568fb414044b2062538389/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/8768df6318693e8df0beb7e78cb18abd090d5e16076ec67337206fa1cec9077a/merged 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:13:00.800 INFO:teuthology.orchestra.run.smithi102.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-24T23:13:00.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:13:00.801 DEBUG:teuthology.orchestra.run.smithi102:> ls /sys/fs/fuse/connections 2024-04-24T23:13:00.853 INFO:teuthology.orchestra.run.smithi102.stdout:115 2024-04-24T23:13:00.854 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-24T23:13:00.854 DEBUG:teuthology.orchestra.run.smithi102:> sudo stdin-killer -- python3 -c ' 2024-04-24T23:13:00.854 DEBUG:teuthology.orchestra.run.smithi102:> import glob 2024-04-24T23:13:00.855 DEBUG:teuthology.orchestra.run.smithi102:> import re 2024-04-24T23:13:00.855 DEBUG:teuthology.orchestra.run.smithi102:> import os 2024-04-24T23:13:00.855 DEBUG:teuthology.orchestra.run.smithi102:> import subprocess 2024-04-24T23:13:00.855 DEBUG:teuthology.orchestra.run.smithi102:> 2024-04-24T23:13:00.855 DEBUG:teuthology.orchestra.run.smithi102:> def _find_admin_socket(client_name): 2024-04-24T23:13:00.855 DEBUG:teuthology.orchestra.run.smithi102:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> files = glob.glob(asok_path) 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> # Given a non-glob path, it better be there 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> if "*" not in asok_path: 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> assert(len(files) == 1) 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> return files[0] 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> for f in files: 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T23:13:00.856 DEBUG:teuthology.orchestra.run.smithi102:> contents = proc_f.read() 2024-04-24T23:13:00.857 DEBUG:teuthology.orchestra.run.smithi102:> if mountpoint in contents: 2024-04-24T23:13:00.857 DEBUG:teuthology.orchestra.run.smithi102:> return f 2024-04-24T23:13:00.857 DEBUG:teuthology.orchestra.run.smithi102:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T23:13:00.857 DEBUG:teuthology.orchestra.run.smithi102:> 2024-04-24T23:13:00.857 DEBUG:teuthology.orchestra.run.smithi102:> print(_find_admin_socket("client.0")) 2024-04-24T23:13:00.857 DEBUG:teuthology.orchestra.run.smithi102:> ' 2024-04-24T23:13:00.972 INFO:teuthology.orchestra.run.smithi102.stdout:/var/run/ceph/ceph-client.0.81764.asok 2024-04-24T23:13:00.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:13:00.982 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.81764.asok 2024-04-24T23:13:00.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:13:00.983 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.81764.asok status 2024-04-24T23:13:01.109 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:13:00 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:13:00.973+0000 7f6892885200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:13:01.126 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:13:01.126 INFO:teuthology.orchestra.run.smithi102.stdout: "metadata": { 2024-04-24T23:13:01.126 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-24T23:13:01.126 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:13:01.126 INFO:teuthology.orchestra.run.smithi102.stdout: "entity_id": "0", 2024-04-24T23:13:01.126 INFO:teuthology.orchestra.run.smithi102.stdout: "hostname": "smithi102", 2024-04-24T23:13:01.126 INFO:teuthology.orchestra.run.smithi102.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "pid": "81764", 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "root": "/volumes/qa/sv_0/77976509-2cd8-41de-8489-e8b8dd738f18" 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "dentry_count": 0, 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "dentry_pinned_count": 0, 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 15312, 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "inst": { 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "name": { 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "client", 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "num": 15312 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": { 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:13:01.127 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 2194420854 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": { 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 2194420854 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "inst_str": "client.15312 192.168.0.1:0/2194420854", 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "addr_str": "192.168.0.1:0/2194420854", 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "inode_count": 5, 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_epoch": 20, 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_epoch": 90, 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_epoch_barrier": 0, 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "blocklisted": false, 2024-04-24T23:13:01.128 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs" 2024-04-24T23:13:01.129 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:13:01.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:13:01.135 DEBUG:teuthology.orchestra.run.smithi102:> stat --file-system '--printf=%T 2024-04-24T23:13:01.135 DEBUG:teuthology.orchestra.run.smithi102:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T23:13:01.151 INFO:teuthology.orchestra.run.smithi102.stdout:fuseblk 2024-04-24T23:13:01.151 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T23:13:01.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:13:01.151 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:13:01.181 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:13:00 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:13:00.900+0000 7f55f5e8c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:13:01.181 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:13:01 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z[35001]: 2024-04-24T23:13:01.115+0000 7f55f5e8c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:13:01.220 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T23:13:01.230 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T23:13:01.230 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:13:01.230 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-24T23:13:01.294 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-mon[29216]: Standby manager daemon z restarted 2024-04-24T23:13:01.294 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-mon[29216]: Standby manager daemon z started 2024-04-24T23:13:01.294 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:13:01.295 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:13:01.295 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:13:01.295 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:13:01.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 -- 172.21.15.102:0/49047687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f441017f0 msgr2=0x7f6f44101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:01.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 --2- 172.21.15.102:0/49047687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f441017f0 0x7f6f44101c10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009900 tx=0x7f6f3802a410 comp rx=0 tx=0).stop 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 -- 172.21.15.102:0/49047687 shutdown_connections 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 --2- 172.21.15.102:0/49047687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f441033a0 0x7f6f44109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 --2- 172.21.15.102:0/49047687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f441029e0 0x7f6f44102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 --2- 172.21.15.102:0/49047687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f441017f0 0x7f6f44101c10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 -- 172.21.15.102:0/49047687 >> 172.21.15.102:0/49047687 conn(0x7f6f440fcd60 msgr2=0x7f6f440ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 -- 172.21.15.102:0/49047687 shutdown_connections 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 -- 172.21.15.102:0/49047687 wait complete. 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.360+0000 7f6f49d90640 1 Processor -- start 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f49d90640 1 -- start start 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f49d90640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f441017f0 0x7f6f441979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f49d90640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f441029e0 0x7f6f44197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f49d90640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f441033a0 0x7f6f4419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f49d90640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4410d360 con 0x7f6f441029e0 2024-04-24T23:13:01.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f49d90640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4410d1e0 con 0x7f6f441017f0 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f49d90640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4410d4e0 con 0x7f6f441033a0 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f42ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f441029e0 0x7f6f44197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f42ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f441029e0 0x7f6f44197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37684/0 (socket says 172.21.15.102:37684) 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f42ffd640 1 -- 172.21.15.102:0/1107429306 learned_addr learned my addr 172.21.15.102:0/1107429306 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f43fff640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f441033a0 0x7f6f4419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f437fe640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f441017f0 0x7f6f441979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f42ffd640 1 -- 172.21.15.102:0/1107429306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f441033a0 msgr2=0x7f6f4419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f42ffd640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f441033a0 0x7f6f4419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f42ffd640 1 -- 172.21.15.102:0/1107429306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f441017f0 msgr2=0x7f6f441979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f42ffd640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f441017f0 0x7f6f441979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f42ffd640 1 -- 172.21.15.102:0/1107429306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4419f5c0 con 0x7f6f441029e0 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f437fe640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f441017f0 0x7f6f441979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:01.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f43fff640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f441033a0 0x7f6f4419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:01.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.361+0000 7f6f42ffd640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f441029e0 0x7f6f44197ee0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400e9f0 tx=0x7f6f3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:01.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.362+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3400cd80 con 0x7f6f441029e0 2024-04-24T23:13:01.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.362+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4419f8b0 con 0x7f6f441029e0 2024-04-24T23:13:01.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.362+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f340054d0 con 0x7f6f441029e0 2024-04-24T23:13:01.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.362+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34010490 con 0x7f6f441029e0 2024-04-24T23:13:01.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.362+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4419fdb0 con 0x7f6f441029e0 2024-04-24T23:13:01.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.363+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f44108b90 con 0x7f6f441029e0 2024-04-24T23:13:01.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.364+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137851+0+0 (secure 0 0 0) 0x7f6f34005000 con 0x7f6f441029e0 2024-04-24T23:13:01.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.365+0000 7f6f40ff9640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6f2c0a2c80 0x7f6f2c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:01.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.365+0000 7f6f437fe640 1 -- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6f2c0a2c80 msgr2=0x7f6f2c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/3611055373 2024-04-24T23:13:01.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.365+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f6f340cf720 con 0x7f6f441029e0 2024-04-24T23:13:01.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.365+0000 7f6f437fe640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6f2c0a2c80 0x7f6f2c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:13:01.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.368+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f34053740 con 0x7f6f441029e0 2024-04-24T23:13:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:01 smithi183 ceph-mon[33137]: Standby manager daemon z restarted 2024-04-24T23:13:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:01 smithi183 ceph-mon[33137]: Standby manager daemon z started 2024-04-24T23:13:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:01 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:13:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:01 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:13:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:01 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:13:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:01 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:13:01.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.536+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f6f340985c0 con 0x7f6f441029e0 2024-04-24T23:13:01.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.536+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6f2c0a2c80 msgr2=0x7f6f2c0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:13:01.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.536+0000 7f6f40ff9640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/3611055373,v1:172.21.15.102:6801/3611055373] conn(0x7f6f2c0a2c80 0x7f6f2c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.292+0000 7f8bf876d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:13:01.572 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.522+0000 7f8bf876d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:13:01.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:01.604+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f6f4402d3c0 con 0x7f6f441029e0 2024-04-24T23:13:01.609 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:13:01 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y[34106]: 2024-04-24T23:13:01.190+0000 7f6892885200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:13:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:01 smithi161 ceph-mon[32709]: Standby manager daemon z restarted 2024-04-24T23:13:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:01 smithi161 ceph-mon[32709]: Standby manager daemon z started 2024-04-24T23:13:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:01 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:13:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:01 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:13:01.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:01 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:13:01.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:01 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.183:0/80992367' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b5a0f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b5a0f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b5a0f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b5a0f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b5a0f6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b578f1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b578f1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b578f1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b578f1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:01.763+0000 7f8b578f1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: Standby manager daemon y restarted 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: Standby manager daemon y started 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: Active manager daemon x restarted 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: Activating manager daemon x 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:13:02.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: mgrmap e21: x(active, starting, since 0.0116546s), standbys: z, y 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:13:02.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:13:02.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: Manager daemon x is now available 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1107429306' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:13:02.516 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:02 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:13:02.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.537+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f6f340035f0 con 0x7f6f441029e0 2024-04-24T23:13:02.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.537+0000 7f6f40ff9640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f2c051930 0x7f6f2c051d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:02.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.539+0000 7f6f437fe640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f2c051930 0x7f6f2c051d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:02.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.545+0000 7f6f437fe640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f2c051930 0x7f6f2c051d10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006010 tx=0x7f6f380314a0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:02.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.546+0000 7f6f40ff9640 1 -- 172.21.15.102:0/1107429306 <== mon.0 v2:172.21.15.102:3300/0 9 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v21) ==== 118+0+0 (secure 0 0 0) 0x7f6f34091690 con 0x7f6f441029e0 2024-04-24T23:13:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.548+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f2c051930 msgr2=0x7f6f2c051d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.548+0000 7f6f49d90640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f2c051930 0x7f6f2c051d10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006010 tx=0x7f6f380314a0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.548+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f441029e0 msgr2=0x7f6f44197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.548+0000 7f6f49d90640 1 --2- 172.21.15.102:0/1107429306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f441029e0 0x7f6f44197ee0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400e9f0 tx=0x7f6f3400edf0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.548+0000 7f6f43fff640 1 -- 172.21.15.102:0/1107429306 reap_dead start 2024-04-24T23:13:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.549+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 shutdown_connections 2024-04-24T23:13:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.549+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 >> 172.21.15.102:0/1107429306 conn(0x7f6f440fcd60 msgr2=0x7f6f44105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.549+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 shutdown_connections 2024-04-24T23:13:02.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.549+0000 7f6f49d90640 1 -- 172.21.15.102:0/1107429306 wait complete. 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: Standby manager daemon y restarted 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: Standby manager daemon y started 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: Active manager daemon x restarted 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: Activating manager daemon x 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:13:02.563 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: mgrmap e21: x(active, starting, since 0.0116546s), standbys: z, y 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:13:02.564 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:13:02.565 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:13:02.565 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:13:02.566 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-24T23:13:02.566 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:13:02.566 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:13:02.566 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:13:02.566 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:13:02.566 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:13:02.566 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:13:02.566 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:13:02.566 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: Manager daemon x is now available 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1107429306' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:13:02.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: Standby manager daemon y restarted 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: Standby manager daemon y started 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.? 172.21.15.161:0/2390138462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: Active manager daemon x restarted 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: Activating manager daemon x 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:13:02.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: mgrmap e21: x(active, starting, since 0.0116546s), standbys: z, y 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:13:02.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:13:02.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: Manager daemon x is now available 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1107429306' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:13:02.613 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:02 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.706+0000 7f2787c6b640 1 -- 172.21.15.102:0/3764909955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780101890 msgr2=0x7f2780101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.706+0000 7f2787c6b640 1 --2- 172.21.15.102:0/3764909955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780101890 0x7f2780101d10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2770009900 tx=0x7f277002a410 comp rx=0 tx=0).stop 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.706+0000 7f2787c6b640 1 -- 172.21.15.102:0/3764909955 shutdown_connections 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.706+0000 7f2787c6b640 1 --2- 172.21.15.102:0/3764909955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2780102250 0x7f2780108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.706+0000 7f2787c6b640 1 --2- 172.21.15.102:0/3764909955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780101890 0x7f2780101d10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.706+0000 7f2787c6b640 1 --2- 172.21.15.102:0/3764909955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27801006a0 0x7f2780100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.706+0000 7f2787c6b640 1 -- 172.21.15.102:0/3764909955 >> 172.21.15.102:0/3764909955 conn(0x7f27800fbc10 msgr2=0x7f27800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.706+0000 7f2787c6b640 1 -- 172.21.15.102:0/3764909955 shutdown_connections 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.706+0000 7f2787c6b640 1 -- 172.21.15.102:0/3764909955 wait complete. 2024-04-24T23:13:02.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2787c6b640 1 Processor -- start 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2787c6b640 1 -- start start 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2787c6b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27801006a0 0x7f2780069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2787c6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2780101890 0x7f278019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2787c6b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780102250 0x7f278019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2787c6b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f278010c1d0 con 0x7f2780102250 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2787c6b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f278010c050 con 0x7f2780101890 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2787c6b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f278010c350 con 0x7f27801006a0 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2786208640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780102250 0x7f278019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2785a07640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27801006a0 0x7f2780069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2786208640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780102250 0x7f278019ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37806/0 (socket says 172.21.15.102:37806) 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2786208640 1 -- 172.21.15.102:0/1107670316 learned_addr learned my addr 172.21.15.102:0/1107670316 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.707+0000 7f2785206640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2780101890 0x7f278019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:02.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2786208640 1 -- 172.21.15.102:0/1107670316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27801006a0 msgr2=0x7f2780069980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2786208640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27801006a0 0x7f2780069980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2786208640 1 -- 172.21.15.102:0/1107670316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2780101890 msgr2=0x7f278019c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2786208640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2780101890 0x7f278019c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2786208640 1 -- 172.21.15.102:0/1107670316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278019f4e0 con 0x7f2780102250 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2785206640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2780101890 0x7f278019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2786208640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780102250 0x7f278019ec20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f277800cce0 tx=0x7f277800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2776ffd640 1 -- 172.21.15.102:0/1107670316 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277800f450 con 0x7f2780102250 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278019f7d0 con 0x7f2780102250 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2776ffd640 1 -- 172.21.15.102:0/1107670316 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f277800faa0 con 0x7f2780102250 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2776ffd640 1 -- 172.21.15.102:0/1107670316 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778018b70 con 0x7f2780102250 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.708+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278019fcd0 con 0x7f2780102250 2024-04-24T23:13:02.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.709+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27801079c0 con 0x7f2780102250 2024-04-24T23:13:02.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.710+0000 7f2776ffd640 1 -- 172.21.15.102:0/1107670316 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f2778018d10 con 0x7f2780102250 2024-04-24T23:13:02.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.710+0000 7f2776ffd640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f276c0a2a40 0x7f276c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:02.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.711+0000 7f2785a07640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f276c0a2a40 0x7f276c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:02.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.711+0000 7f2776ffd640 1 -- 172.21.15.102:0/1107670316 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f27780cf270 con 0x7f2780102250 2024-04-24T23:13:02.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.711+0000 7f2785a07640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f276c0a2a40 0x7f276c0a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f27801016f0 tx=0x7f277c005f90 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:02.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.714+0000 7f2776ffd640 1 -- 172.21.15.102:0/1107670316 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2778014030 con 0x7f2780102250 2024-04-24T23:13:02.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.926+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 --> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f2780061b80 con 0x7f276c0a2a40 2024-04-24T23:13:02.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.941+0000 7f2776ffd640 1 -- 172.21.15.102:0/1107670316 <== mgr.15288 v2:172.21.15.102:6800/161836032 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f2780061b80 con 0x7f276c0a2a40 2024-04-24T23:13:02.943 INFO:teuthology.orchestra.run.smithi102.stdout:{} 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f276c0a2a40 msgr2=0x7f276c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f276c0a2a40 0x7f276c0a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f27801016f0 tx=0x7f277c005f90 comp rx=0 tx=0).stop 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780102250 msgr2=0x7f278019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780102250 0x7f278019ec20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f277800cce0 tx=0x7f277800a620 comp rx=0 tx=0).stop 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 shutdown_connections 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f276c0a2a40 0x7f276c0a4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2780102250 0x7f278019ec20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2780101890 0x7f278019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 --2- 172.21.15.102:0/1107670316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27801006a0 0x7f2780069980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 >> 172.21.15.102:0/1107670316 conn(0x7f27800fbc10 msgr2=0x7f2780104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:02.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 shutdown_connections 2024-04-24T23:13:02.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:02.943+0000 7f2787c6b640 1 -- 172.21.15.102:0/1107670316 wait complete. 2024-04-24T23:13:02.956 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-24T23:13:02.973 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T23:13:02.983 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T23:13:02.983 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:13:02.983 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-24T23:13:03.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.082+0000 7f46a98a5640 1 -- 172.21.15.102:0/3942665141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 msgr2=0x7f46a4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.082+0000 7f46a98a5640 1 --2- 172.21.15.102:0/3942665141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 0x7f46a4101d10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f468c009900 tx=0x7f468c02a410 comp rx=0 tx=0).stop 2024-04-24T23:13:03.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.082+0000 7f46a98a5640 1 -- 172.21.15.102:0/3942665141 shutdown_connections 2024-04-24T23:13:03.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.082+0000 7f46a98a5640 1 --2- 172.21.15.102:0/3942665141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a4102250 0x7f46a4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.082+0000 7f46a98a5640 1 --2- 172.21.15.102:0/3942665141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 0x7f46a4101d10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.082+0000 7f46a98a5640 1 --2- 172.21.15.102:0/3942665141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46a41006a0 0x7f46a4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.082+0000 7f46a98a5640 1 -- 172.21.15.102:0/3942665141 >> 172.21.15.102:0/3942665141 conn(0x7f46a40fbc10 msgr2=0x7f46a40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.082+0000 7f46a98a5640 1 -- 172.21.15.102:0/3942665141 shutdown_connections 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.082+0000 7f46a98a5640 1 -- 172.21.15.102:0/3942665141 wait complete. 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a98a5640 1 Processor -- start 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a98a5640 1 -- start start 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a98a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a41006a0 0x7f46a4069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a98a5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 0x7f46a419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a98a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46a4102250 0x7f46a419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a98a5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f46a410c1d0 con 0x7f46a4101890 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a98a5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46a410c050 con 0x7f46a41006a0 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a98a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46a410c350 con 0x7f46a4102250 2024-04-24T23:13:03.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a27fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 0x7f46a419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a27fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 0x7f46a419c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37830/0 (socket says 172.21.15.102:37830) 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a27fc640 1 -- 172.21.15.102:0/1848495016 learned_addr learned my addr 172.21.15.102:0/1848495016 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a37fe640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46a4102250 0x7f46a419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.083+0000 7f46a2ffd640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a41006a0 0x7f46a4069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a27fc640 1 -- 172.21.15.102:0/1848495016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46a4102250 msgr2=0x7f46a419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a27fc640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46a4102250 0x7f46a419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a27fc640 1 -- 172.21.15.102:0/1848495016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a41006a0 msgr2=0x7f46a4069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a27fc640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a41006a0 0x7f46a4069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a27fc640 1 -- 172.21.15.102:0/1848495016 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a4196660 con 0x7f46a4101890 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a27fc640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 0x7f46a419c730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f468c004770 tx=0x7f468c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a88a3640 1 -- 172.21.15.102:0/1848495016 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c03f030 con 0x7f46a4101890 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a88a3640 1 -- 172.21.15.102:0/1848495016 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f468c003cb0 con 0x7f46a4101890 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a41968f0 con 0x7f46a4101890 2024-04-24T23:13:03.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a88a3640 1 -- 172.21.15.102:0/1848495016 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c035070 con 0x7f46a4101890 2024-04-24T23:13:03.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.084+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a4196e30 con 0x7f46a4101890 2024-04-24T23:13:03.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.085+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a4198210 con 0x7f46a4101890 2024-04-24T23:13:03.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.086+0000 7f46a88a3640 1 -- 172.21.15.102:0/1848495016 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f468c042030 con 0x7f46a4101890 2024-04-24T23:13:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.087+0000 7f46a88a3640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f46900a2ac0 0x7f46900a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.087+0000 7f46a2ffd640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f46900a2ac0 0x7f46900a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.087+0000 7f46a88a3640 1 -- 172.21.15.102:0/1848495016 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f468c0ef610 con 0x7f46a4101890 2024-04-24T23:13:03.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.087+0000 7f46a2ffd640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f46900a2ac0 0x7f46900a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f469c005fd0 tx=0x7f469c00b390 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:03.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.091+0000 7f46a88a3640 1 -- 172.21.15.102:0/1848495016 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f468c0bc0e0 con 0x7f46a4101890 2024-04-24T23:13:03.184 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:02.931+0000 7f8b550ec700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:03.184 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:02.931+0000 7f8b550ec700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:03.184 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:02.931+0000 7f8b550ec700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:03.184 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:02.931+0000 7f8b550ec700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:03.184 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:13:02 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:13:02.931+0000 7f8b550ec700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:13:03.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.290+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 --> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f46a41077c0 con 0x7f46900a2ac0 2024-04-24T23:13:03.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.354+0000 7f46a88a3640 1 -- 172.21.15.102:0/1848495016 <== mgr.15288 v2:172.21.15.102:6800/161836032 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f46a41077c0 con 0x7f46900a2ac0 2024-04-24T23:13:03.356 INFO:teuthology.orchestra.run.smithi102.stdout:Schedule set for path / 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f46900a2ac0 msgr2=0x7f46900a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f46900a2ac0 0x7f46900a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f469c005fd0 tx=0x7f469c00b390 comp rx=0 tx=0).stop 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 msgr2=0x7f46a419c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 0x7f46a419c730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f468c004770 tx=0x7f468c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 shutdown_connections 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46a4102250 0x7f46a419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f46900a2ac0 0x7f46900a4f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4101890 0x7f46a419c730 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 --2- 172.21.15.102:0/1848495016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a41006a0 0x7f46a4069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 >> 172.21.15.102:0/1848495016 conn(0x7f46a40fbc10 msgr2=0x7f46a4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:03.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 shutdown_connections 2024-04-24T23:13:03.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.356+0000 7f46a98a5640 1 -- 172.21.15.102:0/1848495016 wait complete. 2024-04-24T23:13:03.370 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-24T23:13:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.477+0000 7f925e8a5640 1 -- 172.21.15.102:0/3280434095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9258102280 msgr2=0x7f9258104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.477+0000 7f925e8a5640 1 --2- 172.21.15.102:0/3280434095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9258102280 0x7f9258104670 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9248009900 tx=0x7f924802a410 comp rx=0 tx=0).stop 2024-04-24T23:13:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.477+0000 7f925e8a5640 1 -- 172.21.15.102:0/3280434095 shutdown_connections 2024-04-24T23:13:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.477+0000 7f925e8a5640 1 --2- 172.21.15.102:0/3280434095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92581074e0 0x7f92581098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.477+0000 7f925e8a5640 1 --2- 172.21.15.102:0/3280434095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9258104bb0 0x7f9258106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.477+0000 7f925e8a5640 1 --2- 172.21.15.102:0/3280434095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9258102280 0x7f9258104670 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.477+0000 7f925e8a5640 1 -- 172.21.15.102:0/3280434095 >> 172.21.15.102:0/3280434095 conn(0x7f92580fbc10 msgr2=0x7f92580fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.477+0000 7f925e8a5640 1 -- 172.21.15.102:0/3280434095 shutdown_connections 2024-04-24T23:13:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.477+0000 7f925e8a5640 1 -- 172.21.15.102:0/3280434095 wait complete. 2024-04-24T23:13:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925e8a5640 1 Processor -- start 2024-04-24T23:13:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925e8a5640 1 -- start start 2024-04-24T23:13:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925e8a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9258102280 0x7f9258197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925e8a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9258104bb0 0x7f9258197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925e8a5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92581074e0 0x7f925819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925e8a5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f925810d2f0 con 0x7f92581074e0 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925e8a5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f925810d170 con 0x7f9258104bb0 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925e8a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f925810d470 con 0x7f9258102280 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925ce42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92581074e0 0x7f925819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925ce42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92581074e0 0x7f925819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37840/0 (socket says 172.21.15.102:37840) 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925ce42640 1 -- 172.21.15.102:0/2105368694 learned_addr learned my addr 172.21.15.102:0/2105368694 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f9257fff640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9258102280 0x7f9258197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f92577fe640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9258104bb0 0x7f9258197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925ce42640 1 -- 172.21.15.102:0/2105368694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9258102280 msgr2=0x7f9258197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925ce42640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9258102280 0x7f9258197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925ce42640 1 -- 172.21.15.102:0/2105368694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9258104bb0 msgr2=0x7f9258197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925ce42640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9258104bb0 0x7f9258197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.478+0000 7f925ce42640 1 -- 172.21.15.102:0/2105368694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925819f550 con 0x7f92581074e0 2024-04-24T23:13:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.479+0000 7f925ce42640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92581074e0 0x7f925819eee0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f925000eb10 tx=0x7f925000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.479+0000 7f92557fa640 1 -- 172.21.15.102:0/2105368694 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250004b20 con 0x7f92581074e0 2024-04-24T23:13:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.479+0000 7f92557fa640 1 -- 172.21.15.102:0/2105368694 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9250004cc0 con 0x7f92581074e0 2024-04-24T23:13:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.479+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925819f840 con 0x7f92581074e0 2024-04-24T23:13:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.479+0000 7f92557fa640 1 -- 172.21.15.102:0/2105368694 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250010690 con 0x7f92581074e0 2024-04-24T23:13:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.479+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925819fd90 con 0x7f92581074e0 2024-04-24T23:13:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.480+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92580fd700 con 0x7f92581074e0 2024-04-24T23:13:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.481+0000 7f92557fa640 1 -- 172.21.15.102:0/2105368694 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f9250005020 con 0x7f92581074e0 2024-04-24T23:13:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.482+0000 7f92557fa640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92400a2ac0 0x7f92400a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.482+0000 7f9257fff640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92400a2ac0 0x7f92400a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.482+0000 7f92557fa640 1 -- 172.21.15.102:0/2105368694 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f92500cf260 con 0x7f92581074e0 2024-04-24T23:13:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.482+0000 7f9257fff640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92400a2ac0 0x7f92400a4f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9248006010 tx=0x7f9248000bc0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.487+0000 7f92557fa640 1 -- 172.21.15.102:0/2105368694 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f925009c380 con 0x7f92581074e0 2024-04-24T23:13:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:03 smithi183 ceph-mon[33137]: [24/Apr/2024:23:13:02] ENGINE Bus STARTING 2024-04-24T23:13:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:03 smithi183 ceph-mon[33137]: mgrmap e22: x(active, since 1.01782s), standbys: z, y 2024-04-24T23:13:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:03 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1107429306' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T23:13:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:03 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:13:03.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:03 smithi102 ceph-mon[29216]: [24/Apr/2024:23:13:02] ENGINE Bus STARTING 2024-04-24T23:13:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:03 smithi102 ceph-mon[29216]: mgrmap e22: x(active, since 1.01782s), standbys: z, y 2024-04-24T23:13:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:03 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1107429306' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T23:13:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:03 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:13:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:03 smithi161 ceph-mon[32709]: [24/Apr/2024:23:13:02] ENGINE Bus STARTING 2024-04-24T23:13:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:03 smithi161 ceph-mon[32709]: mgrmap e22: x(active, since 1.01782s), standbys: z, y 2024-04-24T23:13:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:03 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1107429306' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T23:13:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:03 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:13:03.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.742+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 --> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f9258061c20 con 0x7f92400a2ac0 2024-04-24T23:13:03.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.802+0000 7f92557fa640 1 -- 172.21.15.102:0/2105368694 <== mgr.15288 v2:172.21.15.102:6800/161836032 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f9258061c20 con 0x7f92400a2ac0 2024-04-24T23:13:03.804 INFO:teuthology.orchestra.run.smithi102.stdout:Retention added to path / 2024-04-24T23:13:03.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92400a2ac0 msgr2=0x7f92400a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92400a2ac0 0x7f92400a4f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9248006010 tx=0x7f9248000bc0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92581074e0 msgr2=0x7f925819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92581074e0 0x7f925819eee0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f925000eb10 tx=0x7f925000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 shutdown_connections 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92400a2ac0 0x7f92400a4f80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92581074e0 0x7f925819eee0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9258104bb0 0x7f9258197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 --2- 172.21.15.102:0/2105368694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9258102280 0x7f9258197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.804+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 >> 172.21.15.102:0/2105368694 conn(0x7f92580fbc10 msgr2=0x7f9258102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.805+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 shutdown_connections 2024-04-24T23:13:03.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.805+0000 7f925e8a5640 1 -- 172.21.15.102:0/2105368694 wait complete. 2024-04-24T23:13:03.817 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 -- 172.21.15.102:0/717032375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4101890 msgr2=0x7f8ca4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/717032375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4101890 0x7f8ca4101d10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009900 tx=0x7f8c9802a410 comp rx=0 tx=0).stop 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 -- 172.21.15.102:0/717032375 shutdown_connections 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/717032375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca4102250 0x7f8ca4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/717032375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4101890 0x7f8ca4101d10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/717032375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ca41006a0 0x7f8ca4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 -- 172.21.15.102:0/717032375 >> 172.21.15.102:0/717032375 conn(0x7f8ca40fbc10 msgr2=0x7f8ca40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 -- 172.21.15.102:0/717032375 shutdown_connections 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 -- 172.21.15.102:0/717032375 wait complete. 2024-04-24T23:13:03.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 Processor -- start 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.923+0000 7f8ca9f63640 1 -- start start 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca9f63640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca41006a0 0x7f8ca4199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca9f63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ca4101890 0x7f8ca419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca9f63640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4102250 0x7f8ca41a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca9f63640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410d350 con 0x7f8ca4102250 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca9f63640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410d1d0 con 0x7f8ca4101890 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca9f63640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410d4d0 con 0x7f8ca41006a0 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4102250 0x7f8ca41a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca41006a0 0x7f8ca4199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ca4101890 0x7f8ca419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca41006a0 0x7f8ca4199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:37620/0 (socket says 172.21.15.102:37620) 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca37fe640 1 -- 172.21.15.102:0/863428132 learned_addr learned my addr 172.21.15.102:0/863428132 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4102250 0x7f8ca41a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37878/0 (socket says 172.21.15.102:37878) 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca2ffd640 1 -- 172.21.15.102:0/863428132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca41006a0 msgr2=0x7f8ca4199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca2ffd640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca41006a0 0x7f8ca4199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca2ffd640 1 -- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4102250 msgr2=0x7f8ca41a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca2ffd640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4102250 0x7f8ca41a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca2ffd640 1 -- 172.21.15.102:0/863428132 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca41a1730 con 0x7f8ca4101890 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca37fe640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca41006a0 0x7f8ca4199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca3fff640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4102250 0x7f8ca41a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca2ffd640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ca4101890 0x7f8ca419a050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8c9802a930 tx=0x7f8c98003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca0ff9640 1 -- 172.21.15.102:0/863428132 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9803f070 con 0x7f8ca4101890 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.924+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca41a19c0 con 0x7f8ca4101890 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.925+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca41a1ed0 con 0x7f8ca4101890 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.925+0000 7f8ca0ff9640 1 -- 172.21.15.102:0/863428132 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c98031c40 con 0x7f8ca4101890 2024-04-24T23:13:03.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.925+0000 7f8ca0ff9640 1 -- 172.21.15.102:0/863428132 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98049060 con 0x7f8ca4101890 2024-04-24T23:13:03.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.925+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca404f500 con 0x7f8ca4101890 2024-04-24T23:13:03.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.927+0000 7f8ca0ff9640 1 -- 172.21.15.102:0/863428132 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f8c98042070 con 0x7f8ca4101890 2024-04-24T23:13:03.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.927+0000 7f8ca0ff9640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8c8c0a2a40 0x7f8c8c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:03.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.927+0000 7f8ca37fe640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8c8c0a2a40 0x7f8c8c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:03.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.927+0000 7f8ca0ff9640 1 -- 172.21.15.102:0/863428132 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f8c980ef9c0 con 0x7f8ca4101890 2024-04-24T23:13:03.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.928+0000 7f8ca37fe640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8c8c0a2a40 0x7f8c8c0a4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8c940058c0 tx=0x7f8c94005f90 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:03.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:03.935+0000 7f8ca0ff9640 1 -- 172.21.15.102:0/863428132 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8c980bc490 con 0x7f8ca4101890 2024-04-24T23:13:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.139+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 --> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f8ca410abc0 con 0x7f8c8c0a2a40 2024-04-24T23:13:04.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.146+0000 7f8ca0ff9640 1 -- 172.21.15.102:0/863428132 <== mgr.15288 v2:172.21.15.102:6800/161836032 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f8ca410abc0 con 0x7f8c8c0a2a40 2024-04-24T23:13:04.148 INFO:teuthology.orchestra.run.smithi102.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T23:13:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-24T23:13:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8c8c0a2a40 msgr2=0x7f8c8c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8c8c0a2a40 0x7f8c8c0a4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8c940058c0 tx=0x7f8c94005f90 comp rx=0 tx=0).stop 2024-04-24T23:13:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ca4101890 msgr2=0x7f8ca419a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ca4101890 0x7f8ca419a050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8c9802a930 tx=0x7f8c98003c10 comp rx=0 tx=0).stop 2024-04-24T23:13:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 shutdown_connections 2024-04-24T23:13:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8c8c0a2a40 0x7f8c8c0a4f00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4102250 0x7f8ca41a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ca4101890 0x7f8ca419a050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 --2- 172.21.15.102:0/863428132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ca41006a0 0x7f8ca4199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 >> 172.21.15.102:0/863428132 conn(0x7f8ca40fbc10 msgr2=0x7f8ca41094a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 shutdown_connections 2024-04-24T23:13:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.148+0000 7f8ca9f63640 1 -- 172.21.15.102:0/863428132 wait complete. 2024-04-24T23:13:04.161 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T23:13:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.268+0000 7f82b294f640 1 -- 172.21.15.102:0/482794483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 msgr2=0x7f82ac100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.268+0000 7f82b294f640 1 --2- 172.21.15.102:0/482794483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 0x7f82ac100490 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f829c009900 tx=0x7f829c02a410 comp rx=0 tx=0).stop 2024-04-24T23:13:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.268+0000 7f82b294f640 1 -- 172.21.15.102:0/482794483 shutdown_connections 2024-04-24T23:13:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.268+0000 7f82b294f640 1 --2- 172.21.15.102:0/482794483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82ac101370 0x7f82ac1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.268+0000 7f82b294f640 1 --2- 172.21.15.102:0/482794483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac1009d0 0x7f82ac100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.268+0000 7f82b294f640 1 --2- 172.21.15.102:0/482794483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 0x7f82ac100490 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.268+0000 7f82b294f640 1 -- 172.21.15.102:0/482794483 >> 172.21.15.102:0/482794483 conn(0x7f82ac0fbc10 msgr2=0x7f82ac0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:04.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.268+0000 7f82b294f640 1 -- 172.21.15.102:0/482794483 shutdown_connections 2024-04-24T23:13:04.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.268+0000 7f82b294f640 1 -- 172.21.15.102:0/482794483 wait complete. 2024-04-24T23:13:04.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82b294f640 1 Processor -- start 2024-04-24T23:13:04.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82b294f640 1 -- start start 2024-04-24T23:13:04.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82b294f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 0x7f82ac199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:04.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82b294f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac1009d0 0x7f82ac19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:04.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82b294f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82ac101370 0x7f82ac1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:04.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82b294f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac10f490 con 0x7f82ac100070 2024-04-24T23:13:04.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82b294f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac10f310 con 0x7f82ac101370 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82b294f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac10f610 con 0x7f82ac1009d0 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82abfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 0x7f82ac199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82abfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 0x7f82ac199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37890/0 (socket says 172.21.15.102:37890) 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82abfff640 1 -- 172.21.15.102:0/746395664 learned_addr learned my addr 172.21.15.102:0/746395664 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82b0eec640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82ac101370 0x7f82ac1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82ab7fe640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac1009d0 0x7f82ac19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82abfff640 1 -- 172.21.15.102:0/746395664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac1009d0 msgr2=0x7f82ac19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82abfff640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac1009d0 0x7f82ac19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82abfff640 1 -- 172.21.15.102:0/746395664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82ac101370 msgr2=0x7f82ac1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82abfff640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82ac101370 0x7f82ac1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82abfff640 1 -- 172.21.15.102:0/746395664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac1a1760 con 0x7f82ac100070 2024-04-24T23:13:04.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.269+0000 7f82ab7fe640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac1009d0 0x7f82ac19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:04.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.270+0000 7f82abfff640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 0x7f82ac199b40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f829c005940 tx=0x7f829c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:04.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.270+0000 7f82a97fa640 1 -- 172.21.15.102:0/746395664 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c03f030 con 0x7f82ac100070 2024-04-24T23:13:04.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.270+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac1a19f0 con 0x7f82ac100070 2024-04-24T23:13:04.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.270+0000 7f82a97fa640 1 -- 172.21.15.102:0/746395664 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f829c003cb0 con 0x7f82ac100070 2024-04-24T23:13:04.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.270+0000 7f82a97fa640 1 -- 172.21.15.102:0/746395664 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c035070 con 0x7f82ac100070 2024-04-24T23:13:04.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.270+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1a1f30 con 0x7f82ac100070 2024-04-24T23:13:04.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.271+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac10adc0 con 0x7f82ac100070 2024-04-24T23:13:04.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.272+0000 7f82a97fa640 1 -- 172.21.15.102:0/746395664 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 23) ==== 138005+0+0 (secure 0 0 0) 0x7f829c042030 con 0x7f82ac100070 2024-04-24T23:13:04.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.273+0000 7f82a97fa640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82940a2e40 0x7f82940a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:04.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.273+0000 7f82ab7fe640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82940a2e40 0x7f82940a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:04.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.273+0000 7f82a97fa640 1 -- 172.21.15.102:0/746395664 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f829c0f0220 con 0x7f82ac100070 2024-04-24T23:13:04.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.273+0000 7f82ab7fe640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82940a2e40 0x7f82940a5300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82a00058c0 tx=0x7f82a0005850 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:04.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.277+0000 7f82a97fa640 1 -- 172.21.15.102:0/746395664 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f829c0bccc0 con 0x7f82ac100070 2024-04-24T23:13:04.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.482+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 --> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f82ac061b80 con 0x7f82940a2e40 2024-04-24T23:13:04.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.489+0000 7f82a97fa640 1 -- 172.21.15.102:0/746395664 <== mgr.15288 v2:172.21.15.102:6800/161836032 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f82ac061b80 con 0x7f82940a2e40 2024-04-24T23:13:04.491 INFO:teuthology.orchestra.run.smithi102.stdout:/ 1m 6m3h 2024-04-24T23:13:04.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82940a2e40 msgr2=0x7f82940a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:04.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82940a2e40 0x7f82940a5300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82a00058c0 tx=0x7f82a0005850 comp rx=0 tx=0).stop 2024-04-24T23:13:04.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 msgr2=0x7f82ac199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:04.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 0x7f82ac199b40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f829c005940 tx=0x7f829c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 shutdown_connections 2024-04-24T23:13:04.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82ac101370 0x7f82ac1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac1009d0 0x7f82ac19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82940a2e40 0x7f82940a5300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 --2- 172.21.15.102:0/746395664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ac100070 0x7f82ac199b40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:04.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 >> 172.21.15.102:0/746395664 conn(0x7f82ac0fbc10 msgr2=0x7f82ac0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:04.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 shutdown_connections 2024-04-24T23:13:04.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:13:04.491+0000 7f82b294f640 1 -- 172.21.15.102:0/746395664 wait complete. 2024-04-24T23:13:04.503 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: [24/Apr/2024:23:13:02] ENGINE Serving on https://172.21.15.102:7150 2024-04-24T23:13:04.503 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: [24/Apr/2024:23:13:02] ENGINE Serving on http://172.21.15.102:8765 2024-04-24T23:13:04.503 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: [24/Apr/2024:23:13:02] ENGINE Bus STARTED 2024-04-24T23:13:04.503 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:13:04.503 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:04.504 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:04.504 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.504 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.504 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.504 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.504 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.504 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:04 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.505 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-24T23:13:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: [24/Apr/2024:23:13:02] ENGINE Serving on https://172.21.15.102:7150 2024-04-24T23:13:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: [24/Apr/2024:23:13:02] ENGINE Serving on http://172.21.15.102:8765 2024-04-24T23:13:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: [24/Apr/2024:23:13:02] ENGINE Bus STARTED 2024-04-24T23:13:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:13:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:04 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.529 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-24T23:13:04.540 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-24T23:13:04.549 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:13:04.549 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-24T23:13:04.572 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T23:13:04.585 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:13:04.585 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T23:13:04.585 INFO:tasks.workunit:timeout=3h 2024-04-24T23:13:04.585 INFO:tasks.workunit:cleanup=True 2024-04-24T23:13:04.586 DEBUG:teuthology.orchestra.run.smithi102:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T23:13:04.601 INFO:teuthology.orchestra.run.smithi102.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T23:13:04.601 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T23:13:04.601 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 73h/115d Inode: 1 Links: 2 2024-04-24T23:13:04.601 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:13:04.601 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T23:13:04.601 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-24 23:11:05.546077913 +0000 2024-04-24T23:13:04.601 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-24 23:11:05.546077913 +0000 2024-04-24T23:13:04.601 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-24 23:13:01.216389725 +0000 2024-04-24T23:13:04.602 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2024-04-24T23:13:04.602 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T23:13:04.602 DEBUG:teuthology.orchestra.run.smithi102:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T23:13:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: [24/Apr/2024:23:13:02] ENGINE Serving on https://172.21.15.102:7150 2024-04-24T23:13:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: [24/Apr/2024:23:13:02] ENGINE Serving on http://172.21.15.102:8765 2024-04-24T23:13:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: [24/Apr/2024:23:13:02] ENGINE Bus STARTED 2024-04-24T23:13:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:13:04.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:04.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:04.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:04 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:04.671 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:13:04.726 INFO:tasks.workunit.client.0.smithi102.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T23:13:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:13:05.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:05.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:05.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-24T23:13:05.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:05.515 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:05 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:13:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-24T23:13:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:05.575 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:05 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:13:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='client.24899 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:05.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:05.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:05.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:05 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:06.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Unable to set osd_memory_target on smithi102 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:06.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:06.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:06.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:06 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Unable to set osd_memory_target on smithi102 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-24T23:13:06.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:06.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:06 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:06.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Unable to set osd_memory_target on smithi102 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:06.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:06.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:06.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:06.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:06 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:13:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:07 smithi183 ceph-mon[33137]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:07 smithi183 ceph-mon[33137]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:07 smithi183 ceph-mon[33137]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:07 smithi183 ceph-mon[33137]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:13:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:07 smithi183 ceph-mon[33137]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:07 smithi183 ceph-mon[33137]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:07 smithi183 ceph-mon[33137]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:07.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:07 smithi183 ceph-mon[33137]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-24T23:13:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:07 smithi102 ceph-mon[29216]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:07 smithi102 ceph-mon[29216]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:07 smithi102 ceph-mon[29216]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:07 smithi102 ceph-mon[29216]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:13:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:07 smithi102 ceph-mon[29216]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:07 smithi102 ceph-mon[29216]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:07 smithi102 ceph-mon[29216]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:07 smithi102 ceph-mon[29216]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-24T23:13:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:07 smithi161 ceph-mon[32709]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:07 smithi161 ceph-mon[32709]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:07 smithi161 ceph-mon[32709]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:07.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:07 smithi161 ceph-mon[32709]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:13:07.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:07 smithi161 ceph-mon[32709]: Updating smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:07.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:07 smithi161 ceph-mon[32709]: Updating smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:07.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:07 smithi161 ceph-mon[32709]: Updating smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:07.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:07 smithi161 ceph-mon[32709]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-24T23:13:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:09 smithi183 ceph-mon[33137]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-24T23:13:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:09 smithi102 ceph-mon[29216]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-24T23:13:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:09 smithi161 ceph-mon[32709]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-24T23:13:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:11 smithi183 ceph-mon[33137]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-24T23:13:11.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:11 smithi102 ceph-mon[29216]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-24T23:13:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:11 smithi161 ceph-mon[32709]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-24T23:13:13.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:13 smithi183 ceph-mon[33137]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-24T23:13:13.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:13 smithi102 ceph-mon[29216]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-24T23:13:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:13 smithi161 ceph-mon[32709]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-24T23:13:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:15 smithi183 ceph-mon[33137]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-24T23:13:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:15 smithi102 ceph-mon[29216]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-24T23:13:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:15 smithi161 ceph-mon[32709]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-24T23:13:17.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:17 smithi183 ceph-mon[33137]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-24T23:13:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:17 smithi102 ceph-mon[29216]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-24T23:13:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:17 smithi161 ceph-mon[32709]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-24T23:13:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 -- 172.21.15.102:0/3606282542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10106630 msgr2=0x7f5c10106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3606282542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10106630 0x7f5c10106a10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009930 tx=0x7f5c0002a420 comp rx=0 tx=0).stop 2024-04-24T23:13:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 -- 172.21.15.102:0/3606282542 shutdown_connections 2024-04-24T23:13:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3606282542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10101030 0x7f5c1010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3606282542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c10100670 0x7f5c10100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3606282542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10106630 0x7f5c10106a10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 -- 172.21.15.102:0/3606282542 >> 172.21.15.102:0/3606282542 conn(0x7f5c100777c0 msgr2=0x7f5c10077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 -- 172.21.15.102:0/3606282542 shutdown_connections 2024-04-24T23:13:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 -- 172.21.15.102:0/3606282542 wait complete. 2024-04-24T23:13:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 Processor -- start 2024-04-24T23:13:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.704+0000 7f5c169ee640 1 -- start start 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c169ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10100670 0x7f5c100fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c169ee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10101030 0x7f5c100fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c169ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c10106630 0x7f5c100fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c169ee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101108b0 con 0x7f5c10101030 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c169ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10110730 con 0x7f5c10100670 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c169ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10110a30 con 0x7f5c10106630 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10101030 0x7f5c100fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10101030 0x7f5c100fdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56228/0 (socket says 172.21.15.102:56228) 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0f7fe640 1 -- 172.21.15.102:0/3942497098 learned_addr learned my addr 172.21.15.102:0/3942497098 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10100670 0x7f5c100fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c10106630 0x7f5c100fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0f7fe640 1 -- 172.21.15.102:0/3942497098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c10106630 msgr2=0x7f5c100fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c10106630 0x7f5c100fe080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0f7fe640 1 -- 172.21.15.102:0/3942497098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10100670 msgr2=0x7f5c100fd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10100670 0x7f5c100fd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0f7fe640 1 -- 172.21.15.102:0/3942497098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c101a63c0 con 0x7f5c10101030 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10100670 0x7f5c100fd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10101030 0x7f5c100fdb40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400c890 tx=0x7f5c0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c0d7fa640 1 -- 172.21.15.102:0/3942497098 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004220 con 0x7f5c10101030 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.705+0000 7f5c169ee640 1 -- 172.21.15.102:0/3942497098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c101a66b0 con 0x7f5c10101030 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.706+0000 7f5c169ee640 1 -- 172.21.15.102:0/3942497098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c101a6c50 con 0x7f5c10101030 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.706+0000 7f5c0d7fa640 1 -- 172.21.15.102:0/3942497098 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c040043c0 con 0x7f5c10101030 2024-04-24T23:13:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.706+0000 7f5c0d7fa640 1 -- 172.21.15.102:0/3942497098 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004a00 con 0x7f5c10101030 2024-04-24T23:13:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c0d7fa640 1 -- 172.21.15.102:0/3942497098 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138061+0+0 (secure 0 0 0) 0x7f5c04004ba0 con 0x7f5c10101030 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c197de740 1 Processor -- start 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c197de740 1 -- start start 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c0d7fa640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5bf80a2ec0 0x7f5bf80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c197de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2240 0x55f6cd2d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c197de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c197de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2df620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c197de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd236cb0 con 0x55f6cd2dd230 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c197de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd1df5b0 con 0x55f6cd2d2240 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c197de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd232c70 con 0x55f6cd2d2b60 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.708+0000 7f5c0d7fa640 1 -- 172.21.15.102:0/3942497098 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5c040cf4b0 con 0x7f5c10101030 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5bf80a2ec0 0x7f5bf80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2df620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2df620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56238/0 (socket says 172.21.15.102:56238) 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 -- 172.21.15.102:0/1418177327 learned_addr learned my addr 172.21.15.102:0/1418177327 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/1418177327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2240 0x55f6cd2d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/1418177327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 -- 172.21.15.102:0/1418177327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2b60 msgr2=0x55f6cd2dccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/1418177327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2dccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 -- 172.21.15.102:0/1418177327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2240 msgr2=0x55f6cd2d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/1418177327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2240 0x55f6cd2d2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 -- 172.21.15.102:0/1418177327 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6ccf4b210 con 0x55f6cd2dd230 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5bf80a2ec0 0x7f5bf80a5380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800afd0 tx=0x7f5c0800ac60 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/1418177327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2df620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800fbb0 tx=0x7f5c08019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23b0a103b3a1e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5bf67fc640 1 -- 172.21.15.102:0/1418177327 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800dde0 con 0x55f6cd2dd230 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5bf67fc640 1 -- 172.21.15.102:0/1418177327 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c08011470 con 0x55f6cd2dd230 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.709+0000 7f5bf67fc640 1 -- 172.21.15.102:0/1418177327 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0801e660 con 0x55f6cd2dd230 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 -- 172.21.15.102:0/1418177327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 msgr2=0x55f6cd2df620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 --2- 172.21.15.102:0/1418177327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2df620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800fbb0 tx=0x7f5c08019000 comp rx=0 tx=0).stop 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 -- 172.21.15.102:0/1418177327 shutdown_connections 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 --2- 172.21.15.102:0/1418177327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2df620 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 --2- 172.21.15.102:0/1418177327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2dccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 --2- 172.21.15.102:0/1418177327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2240 0x55f6cd2d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 -- 172.21.15.102:0/1418177327 >> 172.21.15.102:0/1418177327 conn(0x55f6cd2ce560 msgr2=0x55f6cd2ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 -- 172.21.15.102:0/1418177327 shutdown_connections 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 -- 172.21.15.102:0/1418177327 wait complete. 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.710+0000 7f5c197de740 1 -- start start 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c197de740 1 Processor -- start 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c197de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2da1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c197de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2da710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c197de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2dac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c197de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd2dffc0 con 0x55f6cd2dd230 2024-04-24T23:13:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c197de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd1f2b50 con 0x55f6cd2d2b60 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c197de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd2e0140 con 0x55f6cd2d2240 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c197de740 10 client.0 _ll_register_callbacks cb 0x55f6cd22dca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c14f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2dac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c14f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2dac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56254/0 (socket says 172.21.15.102:56254) 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c14f8b640 1 -- 172.21.15.102:0/3786971802 learned_addr learned my addr 172.21.15.102:0/3786971802 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2da1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2da710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c14f8b640 1 -- 172.21.15.102:0/3786971802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 msgr2=0x55f6cd2da1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2da1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c14f8b640 1 -- 172.21.15.102:0/3786971802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 msgr2=0x55f6cd2da710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2da710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c14f8b640 1 -- 172.21.15.102:0/3786971802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cd017620 con 0x55f6cd2dd230 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2da710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2dac50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800a860 tx=0x7f5c08011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a43cf252a9b2278 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.711+0000 7f5bef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3786971802 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08034060 con 0x55f6cd2dd230 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3786971802 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5c0800ddc0 con 0x55f6cd2dd230 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c197de740 1 -- 172.21.15.102:0/3786971802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c08011470 con 0x55f6cd2dd230 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3786971802 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c08004a00 con 0x55f6cd2dd230 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c197de740 10 client.15408 fetch_fsmap learned FSMap version 21 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c197de740 1 -- 172.21.15.102:0/3786971802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f6cd2ec210 con 0x55f6cd2dd230 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3786971802 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c08011470 con 0x55f6cd2dd230 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3786971802 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5c08004ba0 con 0x55f6cd2dd230 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c197de740 10 client.15408 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c197de740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c197de740 20 client.15408 populate_metadata read hostname 'smithi102' 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c197de740 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f6cd2ed620 0x55f6cd2efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c197de740 4 client.15408 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c197de740 1 -- 172.21.15.102:0/3786971802 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6cd2441f0 con 0x55f6cd2ed620 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.712+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f6cd2ed620 0x55f6cd2efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.713+0000 7f5bef7fe640 10 client.15408 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.713+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f6cd2ed620 0x55f6cd2efa10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bb0d72c1f3b6d6d server_cookie=8b9d884178454230 in_seq=0 out_seq=0 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3786971802 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6cd2441f0 con 0x55f6cd2ed620 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5bef7fe640 10 client.15408 handle_command_reply: tid=0 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 client.15408 shutdown 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 -- 172.21.15.102:0/3786971802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 msgr2=0x55f6cd2dac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2dac50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800a860 tx=0x7f5c08011b20 comp rx=0 tx=0).stop 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 -- 172.21.15.102:0/3786971802 shutdown_connections 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f6cd2ed620 0x55f6cd2efa10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2dac50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2da710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5bef7fe640 0 client.15408 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 --2- 172.21.15.102:0/3786971802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2da1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 -- 172.21.15.102:0/3786971802 >> 172.21.15.102:0/3786971802 conn(0x55f6cd2ce560 msgr2=0x55f6cd2d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 -- 172.21.15.102:0/3786971802 shutdown_connections 2024-04-24T23:13:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 1 -- 172.21.15.102:0/3786971802 wait complete. 2024-04-24T23:13:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.714+0000 7f5c197de740 20 client.15408 trim_cache size 0 max 16384 2024-04-24T23:13:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c197de740 1 Processor -- start 2024-04-24T23:13:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c197de740 1 -- start start 2024-04-24T23:13:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c197de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c197de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2dc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c197de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c197de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101b1e90 con 0x55f6cd2dd230 2024-04-24T23:13:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c197de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101af760 con 0x55f6cd2d2b60 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c197de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101ad5d0 con 0x55f6cd2d2240 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c14f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c14f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56268/0 (socket says 172.21.15.102:56268) 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c14f8b640 1 -- 172.21.15.102:0/2801345875 learned_addr learned my addr 172.21.15.102:0/2801345875 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/2801345875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2dc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c14f8b640 1 -- 172.21.15.102:0/2801345875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 msgr2=0x55f6cd2cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/2801345875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c14f8b640 1 -- 172.21.15.102:0/2801345875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 msgr2=0x55f6cd2dc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/2801345875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2dc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c14f8b640 1 -- 172.21.15.102:0/2801345875 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cd2d2040 con 0x55f6cd2dd230 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/2801345875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e8ed0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5c08000f80 tx=0x7f5c0800f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728ddbe2b48834b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5bf5ffb640 1 -- 172.21.15.102:0/2801345875 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08034040 con 0x55f6cd2dd230 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5bf5ffb640 1 -- 172.21.15.102:0/2801345875 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c0800fa20 con 0x55f6cd2dd230 2024-04-24T23:13:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.729+0000 7f5bf5ffb640 1 -- 172.21.15.102:0/2801345875 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08012300 con 0x55f6cd2dd230 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 -- 172.21.15.102:0/2801345875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 msgr2=0x55f6cd2e8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 --2- 172.21.15.102:0/2801345875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e8ed0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5c08000f80 tx=0x7f5c0800f7b0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 -- 172.21.15.102:0/2801345875 shutdown_connections 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 --2- 172.21.15.102:0/2801345875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e8ed0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 --2- 172.21.15.102:0/2801345875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2dc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 --2- 172.21.15.102:0/2801345875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 -- 172.21.15.102:0/2801345875 >> 172.21.15.102:0/2801345875 conn(0x55f6cd2ce560 msgr2=0x55f6cd2dc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 -- 172.21.15.102:0/2801345875 shutdown_connections 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 -- 172.21.15.102:0/2801345875 wait complete. 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.730+0000 7f5c197de740 1 -- start start 2024-04-24T23:13:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c197de740 1 Processor -- start 2024-04-24T23:13:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c197de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c197de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c197de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c197de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101adc20 con 0x55f6cd2d2b60 2024-04-24T23:13:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c197de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101af090 con 0x55f6cd2dd230 2024-04-24T23:13:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c197de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101ae2f0 con 0x55f6cd2d2240 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c197de740 10 client.0 _ll_register_callbacks cb 0x7f5c00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2e2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56274/0 (socket says 172.21.15.102:56274) 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0f7fe640 1 -- 172.21.15.102:0/3008591519 learned_addr learned my addr 172.21.15.102:0/3008591519 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0f7fe640 1 -- 172.21.15.102:0/3008591519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 msgr2=0x55f6cd2e56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2e56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0f7fe640 1 -- 172.21.15.102:0/3008591519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2dd230 msgr2=0x55f6cd2e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0f7fe640 1 -- 172.21.15.102:0/3008591519 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cd2e3660 con 0x55f6cd2d2b60 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c14f8b640 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c0f7fe640 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2e2a30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5c0409b700 tx=0x7f5c04097ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c103a51ea8b3d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5bef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3008591519 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0401d040 con 0x55f6cd2d2b60 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3008591519 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5c040050b0 con 0x55f6cd2d2b60 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.731+0000 7f5c197de740 1 -- 172.21.15.102:0/3008591519 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6cd23e540 con 0x55f6cd2d2b60 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3008591519 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c0409fb20 con 0x55f6cd2d2b60 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5c197de740 10 client.15414 fetch_fsmap learned FSMap version 21 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5c197de740 1 -- 172.21.15.102:0/3008591519 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f6cd3537a0 con 0x55f6cd2d2b60 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3008591519 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f6cd23e540 con 0x55f6cd2d2b60 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3008591519 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5c0404c320 con 0x55f6cd2d2b60 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5c197de740 10 client.15414 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5c197de740 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5c197de740 20 client.15414 populate_metadata read hostname 'smithi102' 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5c197de740 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f6cd353f70 0x55f6cd356390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5c197de740 4 client.15414 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:13:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5c197de740 1 -- 172.21.15.102:0/3008591519 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6cd1bc8e0 con 0x55f6cd353f70 2024-04-24T23:13:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.732+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f6cd353f70 0x55f6cd356390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.733+0000 7f5c0ffff640 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f6cd353f70 0x55f6cd356390 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d164e0c7263a6a9 server_cookie=e2e85c6b7e797476 in_seq=0 out_seq=0 2024-04-24T23:13:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.733+0000 7f5bef7fe640 10 client.15414 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5bef7fe640 1 -- 172.21.15.102:0/3008591519 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6cd1bc8e0 con 0x55f6cd353f70 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5bef7fe640 10 client.15414 handle_command_reply: tid=0 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 client.15414 shutdown 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 -- 172.21.15.102:0/3008591519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2d2b60 msgr2=0x55f6cd2e2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2e2a30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5c0409b700 tx=0x7f5c04097ad0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 -- 172.21.15.102:0/3008591519 shutdown_connections 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6cd2dd230 0x55f6cd2e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f6cd353f70 0x55f6cd356390 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cd2d2b60 0x55f6cd2e2a30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5bef7fe640 0 client.15414 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 --2- 172.21.15.102:0/3008591519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6cd2d2240 0x55f6cd2e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 -- 172.21.15.102:0/3008591519 >> 172.21.15.102:0/3008591519 conn(0x55f6cd2ce560 msgr2=0x55f6cd2e7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 -- 172.21.15.102:0/3008591519 shutdown_connections 2024-04-24T23:13:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.744+0000 7f5c197de740 1 -- 172.21.15.102:0/3008591519 wait complete. 2024-04-24T23:13:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c197de740 20 client.15414 trim_cache size 0 max 16384 2024-04-24T23:13:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:13:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:13:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:13:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:13:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 -- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5bf80a2ec0 msgr2=0x7f5bf80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5bf80a2ec0 0x7f5bf80a5380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800afd0 tx=0x7f5c0800ac60 comp rx=0 tx=0).stop 2024-04-24T23:13:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 -- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10101030 msgr2=0x7f5c100fdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10101030 0x7f5c100fdb40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400c890 tx=0x7f5c0400cd50 comp rx=0 tx=0).stop 2024-04-24T23:13:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 -- 172.21.15.102:0/3942497098 shutdown_connections 2024-04-24T23:13:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c10106630 0x7f5c100fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5bf80a2ec0 0x7f5bf80a5380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c10101030 0x7f5c100fdb40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 --2- 172.21.15.102:0/3942497098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10100670 0x7f5c100fd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 -- 172.21.15.102:0/3942497098 >> 172.21.15.102:0/3942497098 conn(0x7f5c100777c0 msgr2=0x7f5c10069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 -- 172.21.15.102:0/3942497098 shutdown_connections 2024-04-24T23:13:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:17.745+0000 7f5c169ee640 1 -- 172.21.15.102:0/3942497098 wait complete. 2024-04-24T23:13:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:19 smithi183 ceph-mon[33137]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-24T23:13:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:19 smithi102 ceph-mon[29216]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-24T23:13:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:19 smithi161 ceph-mon[32709]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-24T23:13:21.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:21 smithi183 ceph-mon[33137]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 49 op/s 2024-04-24T23:13:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:21 smithi102 ceph-mon[29216]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 49 op/s 2024-04-24T23:13:21.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:21 smithi161 ceph-mon[32709]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 49 op/s 2024-04-24T23:13:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:23 smithi183 ceph-mon[33137]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:13:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:23 smithi102 ceph-mon[29216]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:13:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:23 smithi161 ceph-mon[32709]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T23:13:25.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:25 smithi183 ceph-mon[33137]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-24T23:13:25.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:25 smithi102 ceph-mon[29216]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-24T23:13:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:25 smithi161 ceph-mon[32709]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-24T23:13:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:27 smithi183 ceph-mon[33137]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-24T23:13:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:27 smithi102 ceph-mon[29216]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-24T23:13:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:27 smithi161 ceph-mon[32709]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-24T23:13:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:29 smithi183 ceph-mon[33137]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-24T23:13:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:29 smithi102 ceph-mon[29216]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-24T23:13:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:29 smithi161 ceph-mon[32709]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-24T23:13:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.839+0000 7eff6ef09640 1 -- 172.21.15.102:0/3434670919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff68069f70 msgr2=0x7eff6806a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.839+0000 7eff6ef09640 1 --2- 172.21.15.102:0/3434670919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff68069f70 0x7eff6806a3f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009900 tx=0x7eff5c02a480 comp rx=0 tx=0).stop 2024-04-24T23:13:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 -- 172.21.15.102:0/3434670919 shutdown_connections 2024-04-24T23:13:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 --2- 172.21.15.102:0/3434670919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff6806a930 0x7eff6810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 --2- 172.21.15.102:0/3434670919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff68069f70 0x7eff6806a3f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 --2- 172.21.15.102:0/3434670919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff68107940 0x7eff68107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 -- 172.21.15.102:0/3434670919 >> 172.21.15.102:0/3434670919 conn(0x7eff68076440 msgr2=0x7eff68076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 -- 172.21.15.102:0/3434670919 shutdown_connections 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 -- 172.21.15.102:0/3434670919 wait complete. 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 Processor -- start 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 -- start start 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff68069f70 0x7eff68197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6806a930 0x7eff681983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff68107940 0x7eff6819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff68111a10 con 0x7eff6806a930 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff68111890 con 0x7eff68069f70 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff6ef09640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff68111b90 con 0x7eff68107940 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff67fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6806a930 0x7eff681983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff67fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6806a930 0x7eff681983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45228/0 (socket says 172.21.15.102:45228) 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.840+0000 7eff67fff640 1 -- 172.21.15.102:0/2352930450 learned_addr learned my addr 172.21.15.102:0/2352930450 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff6cca5640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff68069f70 0x7eff68197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff68107940 0x7eff6819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff67fff640 1 -- 172.21.15.102:0/2352930450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff68107940 msgr2=0x7eff6819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff67fff640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff68107940 0x7eff6819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff67fff640 1 -- 172.21.15.102:0/2352930450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff68069f70 msgr2=0x7eff68197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff67fff640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff68069f70 0x7eff68197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff67fff640 1 -- 172.21.15.102:0/2352930450 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6819ce30 con 0x7eff6806a930 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff67fff640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6806a930 0x7eff681983d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7eff5c006010 tx=0x7eff5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff65ffb640 1 -- 172.21.15.102:0/2352930450 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c03f030 con 0x7eff6806a930 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff6ef09640 1 -- 172.21.15.102:0/2352930450 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6819d0c0 con 0x7eff6806a930 2024-04-24T23:13:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff65ffb640 1 -- 172.21.15.102:0/2352930450 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff5c003aa0 con 0x7eff6806a930 2024-04-24T23:13:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff65ffb640 1 -- 172.21.15.102:0/2352930450 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c035070 con 0x7eff6806a930 2024-04-24T23:13:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.841+0000 7eff6ef09640 1 -- 172.21.15.102:0/2352930450 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff681a49b0 con 0x7eff6806a930 2024-04-24T23:13:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.843+0000 7eff65ffb640 1 -- 172.21.15.102:0/2352930450 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138061+0+0 (secure 0 0 0) 0x7eff5c042030 con 0x7eff6806a930 2024-04-24T23:13:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff71c0b740 1 Processor -- start 2024-04-24T23:13:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff71c0b740 1 -- start start 2024-04-24T23:13:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db93620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 0x55a45db9dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45dba0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff65ffb640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7eff500a2ec0 0x7eff500a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff71c0b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a45daf7cb0 con 0x55a45db93240 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff71c0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a45daa05b0 con 0x55a45db93b60 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff71c0b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a45daf3c70 con 0x55a45db9e230 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff6d4a6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db93620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff6d4a6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db93620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45244/0 (socket says 172.21.15.102:45244) 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff67fff640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7eff500a2ec0 0x7eff500a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff6d4a6640 1 -- 172.21.15.102:0/997203737 learned_addr learned my addr 172.21.15.102:0/997203737 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.844+0000 7eff6cca5640 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 0x55a45db9dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff65ffb640 1 -- 172.21.15.102:0/2352930450 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7eff5c0efc60 con 0x7eff6806a930 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45dba0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff6cca5640 1 -- 172.21.15.102:0/997203737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 msgr2=0x55a45dba0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff6cca5640 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45dba0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff6cca5640 1 -- 172.21.15.102:0/997203737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 msgr2=0x55a45db93620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff6cca5640 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db93620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff6cca5640 1 -- 172.21.15.102:0/997203737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a45d80c210 con 0x55a45db93b60 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db93620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff67fff640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7eff500a2ec0 0x7eff500a5380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0ed220 tx=0x7eff5c08df50 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff6cca5640 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 0x55a45db9dcf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7eff58009c20 tx=0x7eff58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b23ec10522ec1e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff4effd640 1 -- 172.21.15.102:0/997203737 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58004050 con 0x55a45db93b60 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff4effd640 1 -- 172.21.15.102:0/997203737 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff580054d0 con 0x55a45db93b60 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.845+0000 7eff4effd640 1 -- 172.21.15.102:0/997203737 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58004c30 con 0x55a45db93b60 2024-04-24T23:13:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 -- 172.21.15.102:0/997203737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 msgr2=0x55a45db9dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 0x55a45db9dcf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7eff58009c20 tx=0x7eff58011810 comp rx=0 tx=0).stop 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 -- 172.21.15.102:0/997203737 shutdown_connections 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45dba0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 0x55a45db9dcf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 --2- 172.21.15.102:0/997203737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db93620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 -- 172.21.15.102:0/997203737 >> 172.21.15.102:0/997203737 conn(0x55a45db8f560 msgr2=0x55a45db8f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 -- 172.21.15.102:0/997203737 shutdown_connections 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 -- 172.21.15.102:0/997203737 wait complete. 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 -- start start 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.846+0000 7eff71c0b740 1 Processor -- start 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 0x55a45db9b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db9b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45db9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff71c0b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a45dba1020 con 0x55a45db93b60 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff71c0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a45dab3b50 con 0x55a45db93240 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff71c0b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a45dba11a0 con 0x55a45db9e230 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff71c0b740 10 client.0 _ll_register_callbacks cb 0x55a45daeeca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6cca5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db9b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6cca5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db9b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45250/0 (socket says 172.21.15.102:45250) 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6cca5640 1 -- 172.21.15.102:0/3081091798 learned_addr learned my addr 172.21.15.102:0/3081091798 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 0x55a45db9b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45db9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6cca5640 1 -- 172.21.15.102:0/3081091798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 msgr2=0x55a45db9bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6cca5640 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45db9bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6cca5640 1 -- 172.21.15.102:0/3081091798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 msgr2=0x55a45db9b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6cca5640 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 0x55a45db9b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6cca5640 1 -- 172.21.15.102:0/3081091798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a45d8d8620 con 0x55a45db93b60 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff6cca5640 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db9b640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff58000c00 tx=0x7eff58003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa5beda04941c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.847+0000 7eff47fff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff47fff640 1 -- 172.21.15.102:0/3081091798 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58029070 con 0x55a45db93b60 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff47fff640 1 -- 172.21.15.102:0/3081091798 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7eff58003c00 con 0x55a45db93b60 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff71c0b740 1 -- 172.21.15.102:0/3081091798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff580054d0 con 0x55a45db93b60 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff47fff640 1 -- 172.21.15.102:0/3081091798 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7eff58035050 con 0x55a45db93b60 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff71c0b740 10 client.15426 fetch_fsmap learned FSMap version 21 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff71c0b740 1 -- 172.21.15.102:0/3081091798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a45dbad200 con 0x55a45db93b60 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff47fff640 1 -- 172.21.15.102:0/3081091798 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff580054d0 con 0x55a45db93b60 2024-04-24T23:13:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff47fff640 1 -- 172.21.15.102:0/3081091798 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7eff58023430 con 0x55a45db93b60 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff71c0b740 10 client.15426 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff71c0b740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff71c0b740 20 client.15426 populate_metadata read hostname 'smithi102' 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff71c0b740 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a45dbae610 0x55a45dbb0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff71c0b740 4 client.15426 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff71c0b740 1 -- 172.21.15.102:0/3081091798 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a45db051f0 con 0x55a45dbae610 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.848+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a45dbae610 0x55a45dbb0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.849+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a45dbae610 0x55a45dbb0a00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef2eae4c06d83c8 server_cookie=96366bf4c4fbf972 in_seq=0 out_seq=0 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.849+0000 7eff47fff640 10 client.15426 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff47fff640 1 -- 172.21.15.102:0/3081091798 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a45db051f0 con 0x55a45dbae610 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff47fff640 10 client.15426 handle_command_reply: tid=0 2024-04-24T23:13:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 client.15426 shutdown 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 -- 172.21.15.102:0/3081091798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 msgr2=0x55a45db9b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db9b640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff58000c00 tx=0x7eff58003880 comp rx=0 tx=0).stop 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 -- 172.21.15.102:0/3081091798 shutdown_connections 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45db9bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a45dbae610 0x55a45dbb0a00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db9b640 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 --2- 172.21.15.102:0/3081091798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 0x55a45db9b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff47fff640 0 client.15426 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 -- 172.21.15.102:0/3081091798 >> 172.21.15.102:0/3081091798 conn(0x55a45db8f560 msgr2=0x55a45db95260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 -- 172.21.15.102:0/3081091798 shutdown_connections 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 1 -- 172.21.15.102:0/3081091798 wait complete. 2024-04-24T23:13:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.850+0000 7eff71c0b740 20 client.15426 trim_cache size 0 max 16384 2024-04-24T23:13:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff71c0b740 1 Processor -- start 2024-04-24T23:13:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff71c0b740 1 -- start start 2024-04-24T23:13:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 0x55a45dba9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45dba6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff71c0b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff681afb80 con 0x55a45db93b60 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff71c0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff681ad450 con 0x55a45db93240 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff71c0b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff681ab2c0 con 0x55a45db9e230 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6cca5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6cca5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45262/0 (socket says 172.21.15.102:45262) 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6cca5640 1 -- 172.21.15.102:0/395590135 learned_addr learned my addr 172.21.15.102:0/395590135 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/395590135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 0x55a45dba9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6cca5640 1 -- 172.21.15.102:0/395590135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 msgr2=0x55a45dba6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6cca5640 1 --2- 172.21.15.102:0/395590135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45dba6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6cca5640 1 -- 172.21.15.102:0/395590135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 msgr2=0x55a45dba9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6cca5640 1 --2- 172.21.15.102:0/395590135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 0x55a45dba9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6cca5640 1 -- 172.21.15.102:0/395590135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a45db89850 con 0x55a45db93b60 2024-04-24T23:13:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff6cca5640 1 --2- 172.21.15.102:0/395590135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db954e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff58011850 tx=0x7eff58003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647eaa1c5dab2f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff4e7fc640 1 -- 172.21.15.102:0/395590135 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58029040 con 0x55a45db93b60 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff4e7fc640 1 -- 172.21.15.102:0/395590135 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff58009dd0 con 0x55a45db93b60 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.865+0000 7eff4e7fc640 1 -- 172.21.15.102:0/395590135 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58003e60 con 0x55a45db93b60 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 -- 172.21.15.102:0/395590135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 msgr2=0x55a45db954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 --2- 172.21.15.102:0/395590135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db954e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff58011850 tx=0x7eff58003970 comp rx=0 tx=0).stop 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 -- 172.21.15.102:0/395590135 shutdown_connections 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 --2- 172.21.15.102:0/395590135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45dba6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 --2- 172.21.15.102:0/395590135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93b60 0x55a45db954e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 --2- 172.21.15.102:0/395590135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93240 0x55a45dba9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 -- 172.21.15.102:0/395590135 >> 172.21.15.102:0/395590135 conn(0x55a45db8f560 msgr2=0x55a45db8e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 -- 172.21.15.102:0/395590135 shutdown_connections 2024-04-24T23:13:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 -- 172.21.15.102:0/395590135 wait complete. 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 -- start start 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.866+0000 7eff71c0b740 1 Processor -- start 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db973b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 0x55a45db978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff71c0b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45db97e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff71c0b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff681ab910 con 0x55a45db93240 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff71c0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff681acd80 con 0x55a45db93b60 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff71c0b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff681abfe0 con 0x55a45db9e230 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff71c0b740 10 client.0 _ll_register_callbacks cb 0x7eff60009750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6d4a6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6d4a6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45274/0 (socket says 172.21.15.102:45274) 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6d4a6640 1 -- 172.21.15.102:0/2804447427 learned_addr learned my addr 172.21.15.102:0/2804447427 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6cca5640 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 0x55a45db978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6d4a6640 1 -- 172.21.15.102:0/2804447427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 msgr2=0x55a45db97e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45db97e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6d4a6640 1 -- 172.21.15.102:0/2804447427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 msgr2=0x55a45db978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 0x55a45db978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6d4a6640 1 -- 172.21.15.102:0/2804447427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a45db98520 con 0x55a45db93240 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff6d4a6640 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db973b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff60000f80 tx=0x7eff60014c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc122911684745c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff47fff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff47fff640 1 -- 172.21.15.102:0/2804447427 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60014270 con 0x55a45db93240 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff47fff640 1 -- 172.21.15.102:0/2804447427 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7eff60010bc0 con 0x55a45db93240 2024-04-24T23:13:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.867+0000 7eff71c0b740 1 -- 172.21.15.102:0/2804447427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff58009dd0 con 0x55a45db93240 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff47fff640 1 -- 172.21.15.102:0/2804447427 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7eff60032050 con 0x55a45db93240 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff71c0b740 10 client.15432 fetch_fsmap learned FSMap version 21 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff71c0b740 1 -- 172.21.15.102:0/2804447427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a45dc18ef0 con 0x55a45db93240 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff47fff640 1 -- 172.21.15.102:0/2804447427 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff58009dd0 con 0x55a45db93240 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff47fff640 1 -- 172.21.15.102:0/2804447427 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7eff6000f040 con 0x55a45db93240 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff71c0b740 10 client.15432 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff71c0b740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff71c0b740 20 client.15432 populate_metadata read hostname 'smithi102' 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff71c0b740 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a45dc19670 0x55a45dc1ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff71c0b740 4 client.15432 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:13:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff71c0b740 1 -- 172.21.15.102:0/2804447427 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a45daff540 con 0x55a45dc19670 2024-04-24T23:13:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.868+0000 7eff6cca5640 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a45dc19670 0x55a45dc1ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.869+0000 7eff47fff640 10 client.15432 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.869+0000 7eff6cca5640 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a45dc19670 0x55a45dc1ba90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=334f03ec43d9dfcb server_cookie=b4d0594061e894ce in_seq=0 out_seq=0 2024-04-24T23:13:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.871+0000 7eff47fff640 1 -- 172.21.15.102:0/2804447427 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a45daff540 con 0x55a45dc19670 2024-04-24T23:13:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.871+0000 7eff47fff640 10 client.15432 handle_command_reply: tid=0 2024-04-24T23:13:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.871+0000 7eff71c0b740 1 client.15432 shutdown 2024-04-24T23:13:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 -- 172.21.15.102:0/2804447427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 msgr2=0x55a45db973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db973b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff60000f80 tx=0x7eff60014c00 comp rx=0 tx=0).stop 2024-04-24T23:13:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 -- 172.21.15.102:0/2804447427 shutdown_connections 2024-04-24T23:13:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a45db9e230 0x55a45db97e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a45db93b60 0x55a45db978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a45dc19670 0x55a45dc1ba90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff47fff640 0 client.15432 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 --2- 172.21.15.102:0/2804447427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a45db93240 0x55a45db973b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 -- 172.21.15.102:0/2804447427 >> 172.21.15.102:0/2804447427 conn(0x55a45db8f560 msgr2=0x55a45db9b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 -- 172.21.15.102:0/2804447427 shutdown_connections 2024-04-24T23:13:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 1 -- 172.21.15.102:0/2804447427 wait complete. 2024-04-24T23:13:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff71c0b740 20 client.15432 trim_cache size 0 max 16384 2024-04-24T23:13:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:13:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:13:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:13:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff6ef09640 1 -- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7eff500a2ec0 msgr2=0x7eff500a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff6ef09640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7eff500a2ec0 0x7eff500a5380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0ed220 tx=0x7eff5c08df50 comp rx=0 tx=0).stop 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff6ef09640 1 -- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6806a930 msgr2=0x7eff681983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.872+0000 7eff6ef09640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6806a930 0x7eff681983d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7eff5c006010 tx=0x7eff5c004270 comp rx=0 tx=0).stop 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.873+0000 7eff6ef09640 1 -- 172.21.15.102:0/2352930450 shutdown_connections 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.873+0000 7eff6ef09640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff68107940 0x7eff6819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.873+0000 7eff6ef09640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7eff500a2ec0 0x7eff500a5380 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.873+0000 7eff6ef09640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff6806a930 0x7eff681983d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.873+0000 7eff6ef09640 1 --2- 172.21.15.102:0/2352930450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff68069f70 0x7eff68197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.873+0000 7eff6ef09640 1 -- 172.21.15.102:0/2352930450 >> 172.21.15.102:0/2352930450 conn(0x7eff68076440 msgr2=0x7eff680ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.873+0000 7eff6ef09640 1 -- 172.21.15.102:0/2352930450 shutdown_connections 2024-04-24T23:13:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:30.873+0000 7eff6ef09640 1 -- 172.21.15.102:0/2352930450 wait complete. 2024-04-24T23:13:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:31 smithi183 ceph-mon[33137]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-04-24T23:13:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:31 smithi102 ceph-mon[29216]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-04-24T23:13:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:31 smithi161 ceph-mon[32709]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-04-24T23:13:33.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:33 smithi183 ceph-mon[33137]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-24T23:13:33.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:33 smithi102 ceph-mon[29216]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-24T23:13:33.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:33 smithi161 ceph-mon[32709]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-24T23:13:35.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:35 smithi183 ceph-mon[33137]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-24T23:13:35.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:35 smithi102 ceph-mon[29216]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-24T23:13:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:35 smithi161 ceph-mon[32709]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-24T23:13:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:37 smithi183 ceph-mon[33137]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:37.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:37 smithi102 ceph-mon[29216]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:37 smithi161 ceph-mon[32709]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:39.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:39 smithi183 ceph-mon[33137]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:39.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:39 smithi102 ceph-mon[29216]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:39.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:39 smithi161 ceph-mon[32709]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:41 smithi183 ceph-mon[33137]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:41 smithi102 ceph-mon[29216]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:41.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:41 smithi161 ceph-mon[32709]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:43.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:43 smithi102 ceph-mon[29216]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:43.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:43 smithi161 ceph-mon[32709]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:43.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:43 smithi183 ceph-mon[33137]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.960+0000 7fd14fb30640 1 -- 172.21.15.102:0/859569358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 msgr2=0x7fd148100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.960+0000 7fd14fb30640 1 --2- 172.21.15.102:0/859569358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 0x7fd148100af0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd138009930 tx=0x7fd13802a420 comp rx=0 tx=0).stop 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.960+0000 7fd14fb30640 1 -- 172.21.15.102:0/859569358 shutdown_connections 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.960+0000 7fd14fb30640 1 --2- 172.21.15.102:0/859569358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd148101030 0x7fd14810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.960+0000 7fd14fb30640 1 --2- 172.21.15.102:0/859569358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 0x7fd148100af0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.960+0000 7fd14fb30640 1 --2- 172.21.15.102:0/859569358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd148106630 0x7fd148106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.960+0000 7fd14fb30640 1 -- 172.21.15.102:0/859569358 >> 172.21.15.102:0/859569358 conn(0x7fd1480777c0 msgr2=0x7fd148077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.960+0000 7fd14fb30640 1 -- 172.21.15.102:0/859569358 shutdown_connections 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.960+0000 7fd14fb30640 1 -- 172.21.15.102:0/859569358 wait complete. 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14fb30640 1 Processor -- start 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14fb30640 1 -- start start 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14fb30640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 0x7fd148198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14fb30640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd148101030 0x7fd1481992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14fb30640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd148106630 0x7fd148199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14fb30640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd148110940 con 0x7fd148100670 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14fb30640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481107c0 con 0x7fd148106630 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14fb30640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd148110ac0 con 0x7fd148101030 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d8cc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 0x7fd148198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d8cc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 0x7fd148198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50062/0 (socket says 172.21.15.102:50062) 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d8cc640 1 -- 172.21.15.102:0/1932992901 learned_addr learned my addr 172.21.15.102:0/1932992901 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd148106630 0x7fd148199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d0cb640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd148101030 0x7fd1481992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d8cc640 1 -- 172.21.15.102:0/1932992901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd148101030 msgr2=0x7fd1481992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd148101030 0x7fd1481992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d8cc640 1 -- 172.21.15.102:0/1932992901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd148106630 msgr2=0x7fd148199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd148106630 0x7fd148199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d8cc640 1 -- 172.21.15.102:0/1932992901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1481a4160 con 0x7fd148100670 2024-04-24T23:13:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d0cb640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd148101030 0x7fd1481992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.961+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 0x7fd148198d60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd14400c920 tx=0x7fd14400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.962+0000 7fd13effd640 1 -- 172.21.15.102:0/1932992901 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144007c70 con 0x7fd148100670 2024-04-24T23:13:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.962+0000 7fd13effd640 1 -- 172.21.15.102:0/1932992901 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd144007e10 con 0x7fd148100670 2024-04-24T23:13:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.962+0000 7fd14fb30640 1 -- 172.21.15.102:0/1932992901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1481a4450 con 0x7fd148100670 2024-04-24T23:13:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.962+0000 7fd13effd640 1 -- 172.21.15.102:0/1932992901 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144018820 con 0x7fd148100670 2024-04-24T23:13:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.962+0000 7fd14fb30640 1 -- 172.21.15.102:0/1932992901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1481a49a0 con 0x7fd148100670 2024-04-24T23:13:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.964+0000 7fd13effd640 1 -- 172.21.15.102:0/1932992901 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138061+0+0 (secure 0 0 0) 0x7fd1440189c0 con 0x7fd148100670 2024-04-24T23:13:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.964+0000 7fd15283e740 1 Processor -- start 2024-04-24T23:13:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd15283e740 1 -- start start 2024-04-24T23:13:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd13effd640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd1340a2ec0 0x7fd1340a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd15283e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x5603340f7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd15283e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 0x560334101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd13effd640 1 -- 172.21.15.102:0/1932992901 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fd1440cf930 con 0x7fd148100670 2024-04-24T23:13:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd1340a2ec0 0x7fd1340a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd15283e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560334102230 0x560334104620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd15283e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56033405bcb0 con 0x5603340f7240 2024-04-24T23:13:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd15283e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5603340045b0 con 0x560334102230 2024-04-24T23:13:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd15283e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560334057c70 con 0x5603340f7b60 2024-04-24T23:13:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d8cc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x5603340f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d8cc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x5603340f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50066/0 (socket says 172.21.15.102:50066) 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d8cc640 1 -- 172.21.15.102:0/917164797 learned_addr learned my addr 172.21.15.102:0/917164797 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/917164797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560334102230 0x560334104620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d0cb640 1 --2- 172.21.15.102:0/917164797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 0x560334101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d8cc640 1 -- 172.21.15.102:0/917164797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 msgr2=0x560334101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/917164797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 0x560334101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d8cc640 1 -- 172.21.15.102:0/917164797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560334102230 msgr2=0x560334104620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/917164797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560334102230 0x560334104620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d8cc640 1 -- 172.21.15.102:0/917164797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560333d70210 con 0x5603340f7240 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd1340a2ec0 0x7fd1340a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c770 tx=0x7fd14000cb70 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/917164797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x5603340f7620 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd14409b8f0 tx=0x7fd144018d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fae85b736e0522a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.965+0000 7fd133fff640 1 -- 172.21.15.102:0/917164797 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14409ffa0 con 0x5603340f7240 2024-04-24T23:13:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.966+0000 7fd133fff640 1 -- 172.21.15.102:0/917164797 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1440a4860 con 0x5603340f7240 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.966+0000 7fd133fff640 1 -- 172.21.15.102:0/917164797 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1440cd1d0 con 0x5603340f7240 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.966+0000 7fd15283e740 1 -- 172.21.15.102:0/917164797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 msgr2=0x5603340f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.966+0000 7fd15283e740 1 --2- 172.21.15.102:0/917164797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x5603340f7620 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd14409b8f0 tx=0x7fd144018d70 comp rx=0 tx=0).stop 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.967+0000 7fd15283e740 1 -- 172.21.15.102:0/917164797 shutdown_connections 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.967+0000 7fd15283e740 1 --2- 172.21.15.102:0/917164797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560334102230 0x560334104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.967+0000 7fd15283e740 1 --2- 172.21.15.102:0/917164797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 0x560334101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.967+0000 7fd15283e740 1 --2- 172.21.15.102:0/917164797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x5603340f7620 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.967+0000 7fd15283e740 1 -- 172.21.15.102:0/917164797 >> 172.21.15.102:0/917164797 conn(0x5603340f3560 msgr2=0x5603340f3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.967+0000 7fd15283e740 1 -- 172.21.15.102:0/917164797 shutdown_connections 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.967+0000 7fd15283e740 1 -- 172.21.15.102:0/917164797 wait complete. 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.967+0000 7fd15283e740 1 -- start start 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.967+0000 7fd15283e740 1 Processor -- start 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd15283e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7240 0x5603340ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd15283e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 0x5603340ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd15283e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd15283e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560334105020 con 0x560334102230 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd15283e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560334017b50 con 0x5603340f7240 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd15283e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5603341051a0 con 0x5603340f7b60 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd15283e740 10 client.0 _ll_register_callbacks cb 0x560334052ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14e0cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14e0cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50080/0 (socket says 172.21.15.102:50080) 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14e0cd640 1 -- 172.21.15.102:0/1356068721 learned_addr learned my addr 172.21.15.102:0/1356068721 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7240 0x5603340ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14d0cb640 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 0x5603340ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14e0cd640 1 -- 172.21.15.102:0/1356068721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 msgr2=0x5603340ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 0x5603340ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14e0cd640 1 -- 172.21.15.102:0/1356068721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7240 msgr2=0x5603340ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7240 0x5603340ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14e0cd640 1 -- 172.21.15.102:0/1356068721 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560333e3c620 con 0x560334102230 2024-04-24T23:13:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340ffca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd14000a010 tx=0x7fd140004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94eed8ecc60e5fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd1317fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd1317fa640 1 -- 172.21.15.102:0/1356068721 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140022070 con 0x560334102230 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd15283e740 1 -- 172.21.15.102:0/1356068721 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1440a4860 con 0x560334102230 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd1317fa640 1 -- 172.21.15.102:0/1356068721 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fd14001ac60 con 0x560334102230 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.968+0000 7fd1317fa640 1 -- 172.21.15.102:0/1356068721 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd14001ae60 con 0x560334102230 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd15283e740 10 client.15444 fetch_fsmap learned FSMap version 21 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd15283e740 1 -- 172.21.15.102:0/1356068721 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560334111270 con 0x560334102230 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd1317fa640 1 -- 172.21.15.102:0/1356068721 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1440a4860 con 0x560334102230 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd1317fa640 1 -- 172.21.15.102:0/1356068721 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd140026410 con 0x560334102230 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd15283e740 10 client.15444 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd15283e740 10 client.15444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd15283e740 20 client.15444 populate_metadata read hostname 'smithi102' 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd15283e740 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x560334112780 0x560334114b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd15283e740 4 client.15444 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd15283e740 1 -- 172.21.15.102:0/1356068721 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603340691f0 con 0x560334112780 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.969+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x560334112780 0x560334114b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x560334112780 0x560334114b70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c7c2b68061c8bff server_cookie=c92cea08a2c15ecd in_seq=0 out_seq=0 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd1317fa640 10 client.15444 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd1317fa640 1 -- 172.21.15.102:0/1356068721 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603340691f0 con 0x560334112780 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd1317fa640 10 client.15444 handle_command_reply: tid=0 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd15283e740 1 client.15444 shutdown 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd15283e740 1 -- 172.21.15.102:0/1356068721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 msgr2=0x5603340ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd15283e740 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340ffca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd14000a010 tx=0x7fd140004a40 comp rx=0 tx=0).stop 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd15283e740 1 -- 172.21.15.102:0/1356068721 shutdown_connections 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd15283e740 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x560334112780 0x560334114b70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd15283e740 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340ffca0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd1317fa640 0 client.15444 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd15283e740 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7b60 0x5603340ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.970+0000 7fd15283e740 1 --2- 172.21.15.102:0/1356068721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7240 0x5603340ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.971+0000 7fd15283e740 1 -- 172.21.15.102:0/1356068721 >> 172.21.15.102:0/1356068721 conn(0x5603340f3560 msgr2=0x5603340f9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.971+0000 7fd15283e740 1 -- 172.21.15.102:0/1356068721 shutdown_connections 2024-04-24T23:13:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.971+0000 7fd15283e740 1 -- 172.21.15.102:0/1356068721 wait complete. 2024-04-24T23:13:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:43.971+0000 7fd15283e740 20 client.15444 trim_cache size 0 max 16384 2024-04-24T23:13:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.000+0000 7fd15283e740 1 Processor -- start 2024-04-24T23:13:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.000+0000 7fd15283e740 1 -- start start 2024-04-24T23:13:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.000+0000 7fd15283e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x56033410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.000+0000 7fd15283e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.000+0000 7fd15283e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560334102230 0x56033411a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.000+0000 7fd15283e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481afc70 con 0x5603340f7240 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.000+0000 7fd15283e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481ad540 con 0x5603340f7b60 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.000+0000 7fd15283e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481ab3b0 con 0x560334102230 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d8cc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x56033410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d8cc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x56033410dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50088/0 (socket says 172.21.15.102:50088) 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d8cc640 1 -- 172.21.15.102:0/2730175664 learned_addr learned my addr 172.21.15.102:0/2730175664 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d0cb640 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d8cc640 1 -- 172.21.15.102:0/2730175664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560334102230 msgr2=0x56033411a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560334102230 0x56033411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560334102230 0x56033411a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d8cc640 1 -- 172.21.15.102:0/2730175664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 msgr2=0x5603340f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340f99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d8cc640 1 -- 172.21.15.102:0/2730175664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603340fd790 con 0x5603340f7240 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d0cb640 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x56033410dee0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd1440cd000 tx=0x7fd144097fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bb7048fea51c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd1337fe640 1 -- 172.21.15.102:0/2730175664 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144099ef0 con 0x5603340f7240 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd1337fe640 1 -- 172.21.15.102:0/2730175664 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd144098260 con 0x5603340f7240 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.001+0000 7fd1337fe640 1 -- 172.21.15.102:0/2730175664 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1440cd410 con 0x5603340f7240 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 -- 172.21.15.102:0/2730175664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 msgr2=0x56033410dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x56033410dee0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd1440cd000 tx=0x7fd144097fc0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 -- 172.21.15.102:0/2730175664 shutdown_connections 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560334102230 0x56033411a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 --2- 172.21.15.102:0/2730175664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603340f7240 0x56033410dee0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 -- 172.21.15.102:0/2730175664 >> 172.21.15.102:0/2730175664 conn(0x5603340f3560 msgr2=0x5603340f25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 -- 172.21.15.102:0/2730175664 shutdown_connections 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 -- 172.21.15.102:0/2730175664 wait complete. 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.002+0000 7fd15283e740 1 -- start start 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd15283e740 1 Processor -- start 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd15283e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7240 0x5603340fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd15283e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd15283e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd15283e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481aba00 con 0x560334102230 2024-04-24T23:13:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd15283e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481ace70 con 0x5603340f7b60 2024-04-24T23:13:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd15283e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481ac0d0 con 0x5603340f7240 2024-04-24T23:13:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd15283e740 10 client.0 _ll_register_callbacks cb 0x7fd140007f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14e0cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14e0cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50094/0 (socket says 172.21.15.102:50094) 2024-04-24T23:13:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14e0cd640 1 -- 172.21.15.102:0/1481131405 learned_addr learned my addr 172.21.15.102:0/1481131405 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14d0cb640 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7240 0x5603340fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14e0cd640 1 -- 172.21.15.102:0/1481131405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7240 msgr2=0x5603340fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7240 0x5603340fc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14e0cd640 1 -- 172.21.15.102:0/1481131405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 msgr2=0x5603340fc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340fc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14e0cd640 1 -- 172.21.15.102:0/1481131405 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560334177f40 con 0x560334102230 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7240 0x5603340fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.003+0000 7fd14d0cb640 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd14e0cd640 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340fcba0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd140004b90 tx=0x7fd1400266c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439582180acb2edc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd1317fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd1317fa640 1 -- 172.21.15.102:0/1481131405 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140032070 con 0x560334102230 2024-04-24T23:13:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd1317fa640 1 -- 172.21.15.102:0/1481131405 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fd14001ac60 con 0x560334102230 2024-04-24T23:13:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd15283e740 1 -- 172.21.15.102:0/1481131405 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd144098260 con 0x560334102230 2024-04-24T23:13:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd1317fa640 1 -- 172.21.15.102:0/1481131405 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd140022040 con 0x560334102230 2024-04-24T23:13:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd15283e740 10 client.15450 fetch_fsmap learned FSMap version 21 2024-04-24T23:13:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd15283e740 1 -- 172.21.15.102:0/1481131405 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560334179210 con 0x560334102230 2024-04-24T23:13:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd1317fa640 1 -- 172.21.15.102:0/1481131405 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd144098260 con 0x560334102230 2024-04-24T23:13:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd1317fa640 1 -- 172.21.15.102:0/1481131405 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd14001ccc0 con 0x560334102230 2024-04-24T23:13:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd15283e740 10 client.15450 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:13:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd15283e740 10 client.15450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:13:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd15283e740 20 client.15450 populate_metadata read hostname 'smithi102' 2024-04-24T23:13:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd15283e740 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x560334179930 0x56033417bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd15283e740 4 client.15450 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:13:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.004+0000 7fd15283e740 1 -- 172.21.15.102:0/1481131405 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560334063540 con 0x560334179930 2024-04-24T23:13:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.005+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x560334179930 0x56033417bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.005+0000 7fd14d8cc640 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x560334179930 0x56033417bd50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=721f593987b9f46d server_cookie=ebfc54d5565d8a6a in_seq=0 out_seq=0 2024-04-24T23:13:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.005+0000 7fd1317fa640 10 client.15450 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.008+0000 7fd1317fa640 1 -- 172.21.15.102:0/1481131405 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560334063540 con 0x560334179930 2024-04-24T23:13:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.008+0000 7fd1317fa640 10 client.15450 handle_command_reply: tid=0 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.008+0000 7fd15283e740 1 client.15450 shutdown 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 -- 172.21.15.102:0/1481131405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 msgr2=0x5603340fcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340fcba0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd140004b90 tx=0x7fd1400266c0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 -- 172.21.15.102:0/1481131405 shutdown_connections 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x560334179930 0x56033417bd50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560334102230 0x5603340fcba0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5603340f7b60 0x5603340fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd1317fa640 0 client.15450 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 --2- 172.21.15.102:0/1481131405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603340f7240 0x5603340fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 -- 172.21.15.102:0/1481131405 >> 172.21.15.102:0/1481131405 conn(0x5603340f3560 msgr2=0x5603340ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 -- 172.21.15.102:0/1481131405 shutdown_connections 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 1 -- 172.21.15.102:0/1481131405 wait complete. 2024-04-24T23:13:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd15283e740 20 client.15450 trim_cache size 0 max 16384 2024-04-24T23:13:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:13:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:13:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:13:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:13:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd14fb30640 1 -- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd1340a2ec0 msgr2=0x7fd1340a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.009+0000 7fd14fb30640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd1340a2ec0 0x7fd1340a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c770 tx=0x7fd14000cb70 comp rx=0 tx=0).stop 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 -- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 msgr2=0x7fd148198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 0x7fd148198d60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd14400c920 tx=0x7fd14400cde0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 -- 172.21.15.102:0/1932992901 shutdown_connections 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd148106630 0x7fd148199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd148101030 0x7fd1481992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd1340a2ec0 0x7fd1340a5380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 --2- 172.21.15.102:0/1932992901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd148100670 0x7fd148198d60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 -- 172.21.15.102:0/1932992901 >> 172.21.15.102:0/1932992901 conn(0x7fd1480777c0 msgr2=0x7fd1481030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 -- 172.21.15.102:0/1932992901 shutdown_connections 2024-04-24T23:13:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:44.010+0000 7fd14fb30640 1 -- 172.21.15.102:0/1932992901 wait complete. 2024-04-24T23:13:45.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:45 smithi102 ceph-mon[29216]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:45.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:45 smithi161 ceph-mon[32709]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:45.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:45 smithi183 ceph-mon[33137]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:47 smithi102 ceph-mon[29216]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:47 smithi161 ceph-mon[32709]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:47.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:47 smithi183 ceph-mon[33137]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:13:49.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:49 smithi102 ceph-mon[29216]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:49 smithi161 ceph-mon[32709]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:49 smithi183 ceph-mon[33137]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:51 smithi102 ceph-mon[29216]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:51 smithi161 ceph-mon[32709]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:51 smithi183 ceph-mon[33137]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:53 smithi102 ceph-mon[29216]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:53 smithi161 ceph-mon[32709]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:53 smithi183 ceph-mon[33137]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:55 smithi102 ceph-mon[29216]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:55 smithi161 ceph-mon[32709]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:55 smithi183 ceph-mon[33137]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.135+0000 7f0b6a42c640 1 -- 172.21.15.102:0/1685188614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64101030 msgr2=0x7f0b6410d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.135+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/1685188614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64101030 0x7f0b6410d4f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009cd0 tx=0x7f0b5c02a8e0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.135+0000 7f0b6a42c640 1 -- 172.21.15.102:0/1685188614 shutdown_connections 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.135+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/1685188614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64101030 0x7f0b6410d4f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.135+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/1685188614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b64100670 0x7f0b64100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.135+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/1685188614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b64106630 0x7f0b64106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.135+0000 7f0b6a42c640 1 -- 172.21.15.102:0/1685188614 >> 172.21.15.102:0/1685188614 conn(0x7f0b640777c0 msgr2=0x7f0b64077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.135+0000 7f0b6a42c640 1 -- 172.21.15.102:0/1685188614 shutdown_connections 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.135+0000 7f0b6a42c640 1 -- 172.21.15.102:0/1685188614 wait complete. 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b6a42c640 1 Processor -- start 2024-04-24T23:13:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b6a42c640 1 -- start start 2024-04-24T23:13:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b6a42c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b64100670 0x7f0b64197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b6a42c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b64101030 0x7f0b64198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b6a42c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64106630 0x7f0b6419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b6a42c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64110940 con 0x7f0b64106630 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b6a42c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641107c0 con 0x7f0b64101030 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b6a42c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64110ac0 con 0x7f0b64100670 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b689c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64106630 0x7f0b6419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.136+0000 7f0b63fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b64100670 0x7f0b64197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b689c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64106630 0x7f0b6419c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50152/0 (socket says 172.21.15.102:50152) 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b63fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b64100670 0x7f0b64197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:40560/0 (socket says 172.21.15.102:40560) 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b689c9640 1 -- 172.21.15.102:0/3323461138 learned_addr learned my addr 172.21.15.102:0/3323461138 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b637fe640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b64101030 0x7f0b64198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b689c9640 1 -- 172.21.15.102:0/3323461138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b64100670 msgr2=0x7f0b64197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b689c9640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b64100670 0x7f0b64197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b689c9640 1 -- 172.21.15.102:0/3323461138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b64101030 msgr2=0x7f0b64198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b689c9640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b64101030 0x7f0b64198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b689c9640 1 -- 172.21.15.102:0/3323461138 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6419cf40 con 0x7f0b64106630 2024-04-24T23:13:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b689c9640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64106630 0x7f0b6419c810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c005e30 tx=0x7f0b5c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b617fa640 1 -- 172.21.15.102:0/3323461138 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c004240 con 0x7f0b64106630 2024-04-24T23:13:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b6a42c640 1 -- 172.21.15.102:0/3323461138 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6419d1d0 con 0x7f0b64106630 2024-04-24T23:13:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b617fa640 1 -- 172.21.15.102:0/3323461138 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b5c031d00 con 0x7f0b64106630 2024-04-24T23:13:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.137+0000 7f0b6a42c640 1 -- 172.21.15.102:0/3323461138 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b641a4ac0 con 0x7f0b64106630 2024-04-24T23:13:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.138+0000 7f0b617fa640 1 -- 172.21.15.102:0/3323461138 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c030d30 con 0x7f0b64106630 2024-04-24T23:13:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b617fa640 1 -- 172.21.15.102:0/3323461138 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138061+0+0 (secure 0 0 0) 0x7f0b5c0043e0 con 0x7f0b64106630 2024-04-24T23:13:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b6d210740 1 Processor -- start 2024-04-24T23:13:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b6d210740 1 -- start start 2024-04-24T23:13:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b617fa640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0b4c0a2f40 0x7f0b4c0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd30240 0x56256dd30620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 0x56256dd3acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd3b230 0x56256dd3d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b6d210740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56256dc94cb0 con 0x56256dd30b60 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b6d210740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56256dc3d5b0 con 0x56256dd3b230 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.140+0000 7f0b6d210740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56256dc90c70 con 0x56256dd30240 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b617fa640 1 -- 172.21.15.102:0/3323461138 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f0b5c02e030 con 0x7f0b64106630 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b637fe640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0b4c0a2f40 0x7f0b4c0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b63fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd30240 0x56256dd30620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b63fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd30240 0x56256dd30620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:40566/0 (socket says 172.21.15.102:40566) 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b63fff640 1 -- 172.21.15.102:0/3476336883 learned_addr learned my addr 172.21.15.102:0/3476336883 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b689c9640 1 --2- 172.21.15.102:0/3476336883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd3b230 0x56256dd3d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b63fff640 1 -- 172.21.15.102:0/3476336883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd3b230 msgr2=0x56256dd3d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b63fff640 1 --2- 172.21.15.102:0/3476336883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd3b230 0x56256dd3d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b63fff640 1 -- 172.21.15.102:0/3476336883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 msgr2=0x56256dd3acf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:13:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b63fff640 1 --2- 172.21.15.102:0/3476336883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 0x56256dd3acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b63fff640 1 -- 172.21.15.102:0/3476336883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256d9a9210 con 0x56256dd30240 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b637fe640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0b4c0a2f40 0x7f0b4c0a5400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b64077ee0 tx=0x7f0b58009110 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b63fff640 1 --2- 172.21.15.102:0/3476336883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd30240 0x56256dd30620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009cf0 tx=0x7f0b54011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e98f574fdffc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b4a7fc640 1 -- 172.21.15.102:0/3476336883 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54004230 con 0x56256dd30240 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b4a7fc640 1 -- 172.21.15.102:0/3476336883 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b540043d0 con 0x56256dd30240 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.141+0000 7f0b4a7fc640 1 -- 172.21.15.102:0/3476336883 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b540197c0 con 0x56256dd30240 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.142+0000 7f0b6d210740 1 -- 172.21.15.102:0/3476336883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd30240 msgr2=0x56256dd30620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.142+0000 7f0b6d210740 1 --2- 172.21.15.102:0/3476336883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd30240 0x56256dd30620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009cf0 tx=0x7f0b54011810 comp rx=0 tx=0).stop 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.142+0000 7f0b6d210740 1 -- 172.21.15.102:0/3476336883 shutdown_connections 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.142+0000 7f0b6d210740 1 --2- 172.21.15.102:0/3476336883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd3b230 0x56256dd3d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.142+0000 7f0b6d210740 1 --2- 172.21.15.102:0/3476336883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 0x56256dd3acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.142+0000 7f0b6d210740 1 --2- 172.21.15.102:0/3476336883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd30240 0x56256dd30620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.142+0000 7f0b6d210740 1 -- 172.21.15.102:0/3476336883 >> 172.21.15.102:0/3476336883 conn(0x56256dd2c560 msgr2=0x56256dd2c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.142+0000 7f0b6d210740 1 -- 172.21.15.102:0/3476336883 shutdown_connections 2024-04-24T23:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.142+0000 7f0b6d210740 1 -- 172.21.15.102:0/3476336883 wait complete. 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b6d210740 1 -- start start 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b6d210740 1 Processor -- start 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd38180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 0x56256dd386c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd3b230 0x56256dd38c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b6d210740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56256dd3e020 con 0x56256dd30b60 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b6d210740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56256dc50b50 con 0x56256dd30240 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b6d210740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56256dd3e1a0 con 0x56256dd3b230 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b637fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 0x56256dd386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b637fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 0x56256dd386c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50164/0 (socket says 172.21.15.102:50164) 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b637fe640 1 -- 172.21.15.102:0/1106536572 learned_addr learned my addr 172.21.15.102:0/1106536572 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b689c9640 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd3b230 0x56256dd38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b63fff640 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd38180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.143+0000 7f0b6d210740 10 client.0 _ll_register_callbacks cb 0x56256dc8bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b637fe640 1 -- 172.21.15.102:0/1106536572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd3b230 msgr2=0x56256dd38c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b637fe640 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd3b230 0x56256dd38c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b637fe640 1 -- 172.21.15.102:0/1106536572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 msgr2=0x56256dd38180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b637fe640 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd38180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b637fe640 1 -- 172.21.15.102:0/1106536572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256da75620 con 0x56256dd30b60 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b689c9640 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd3b230 0x56256dd38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b63fff640 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd38180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b637fe640 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 0x56256dd386c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009fd0 tx=0x7f0b580047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1dc0e298510de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b437fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b437fe640 1 -- 172.21.15.102:0/1106536572 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58023020 con 0x56256dd30b60 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b437fe640 1 -- 172.21.15.102:0/1106536572 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f0b58013690 con 0x56256dd30b60 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b6d210740 1 -- 172.21.15.102:0/1106536572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b540043d0 con 0x56256dd30b60 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b437fe640 1 -- 172.21.15.102:0/1106536572 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0b5802e050 con 0x56256dd30b60 2024-04-24T23:13:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b6d210740 10 client.15459 fetch_fsmap learned FSMap version 21 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.144+0000 7f0b6d210740 1 -- 172.21.15.102:0/1106536572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56256dd4a2c0 con 0x56256dd30b60 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.145+0000 7f0b437fe640 1 -- 172.21.15.102:0/1106536572 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b540043d0 con 0x56256dd30b60 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.145+0000 7f0b437fe640 1 -- 172.21.15.102:0/1106536572 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0b5801e030 con 0x56256dd30b60 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.145+0000 7f0b6d210740 10 client.15459 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.145+0000 7f0b6d210740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.145+0000 7f0b6d210740 20 client.15459 populate_metadata read hostname 'smithi102' 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.145+0000 7f0b6d210740 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56256dd4bc70 0x56256dd4e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.145+0000 7f0b6d210740 4 client.15459 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.145+0000 7f0b6d210740 1 -- 172.21.15.102:0/1106536572 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56256dca21f0 con 0x56256dd4bc70 2024-04-24T23:13:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.145+0000 7f0b63fff640 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56256dd4bc70 0x56256dd4e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.146+0000 7f0b63fff640 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56256dd4bc70 0x56256dd4e110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99674447afccfdb6 server_cookie=4cc3f2b275d6d470 in_seq=0 out_seq=0 2024-04-24T23:13:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.146+0000 7f0b437fe640 10 client.15459 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b437fe640 1 -- 172.21.15.102:0/1106536572 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56256dca21f0 con 0x56256dd4bc70 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b437fe640 10 client.15459 handle_command_reply: tid=0 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 client.15459 shutdown 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 -- 172.21.15.102:0/1106536572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 msgr2=0x56256dd386c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 0x56256dd386c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009fd0 tx=0x7f0b580047d0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 -- 172.21.15.102:0/1106536572 shutdown_connections 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd3b230 0x56256dd38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56256dd4bc70 0x56256dd4e110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd30b60 0x56256dd386c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b437fe640 0 client.15459 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 --2- 172.21.15.102:0/1106536572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd38180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 -- 172.21.15.102:0/1106536572 >> 172.21.15.102:0/1106536572 conn(0x56256dd2c560 msgr2=0x56256dd32260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 -- 172.21.15.102:0/1106536572 shutdown_connections 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 1 -- 172.21.15.102:0/1106536572 wait complete. 2024-04-24T23:13:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.147+0000 7f0b6d210740 20 client.15459 trim_cache size 0 max 16384 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b6d210740 1 Processor -- start 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b6d210740 1 -- start start 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd46f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 0x56256dd38850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b6d210740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641afb00 con 0x56256dd3b230 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b6d210740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641ad3d0 con 0x56256dd30240 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b6d210740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641ab240 con 0x56256dd329a0 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b637fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b637fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50170/0 (socket says 172.21.15.102:50170) 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b637fe640 1 -- 172.21.15.102:0/217390970 learned_addr learned my addr 172.21.15.102:0/217390970 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b63fff640 1 --2- 172.21.15.102:0/217390970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd46f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b689c9640 1 --2- 172.21.15.102:0/217390970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 0x56256dd38850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b637fe640 1 -- 172.21.15.102:0/217390970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 msgr2=0x56256dd38850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b637fe640 1 --2- 172.21.15.102:0/217390970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 0x56256dd38850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b637fe640 1 -- 172.21.15.102:0/217390970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 msgr2=0x56256dd46f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.162+0000 7f0b637fe640 1 --2- 172.21.15.102:0/217390970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd46f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.163+0000 7f0b637fe640 1 -- 172.21.15.102:0/217390970 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256dd4e790 con 0x56256dd3b230 2024-04-24T23:13:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.163+0000 7f0b637fe640 1 --2- 172.21.15.102:0/217390970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd372e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0b5801fab0 tx=0x7f0b580143e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4a71755194597f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.163+0000 7f0b49ffb640 1 -- 172.21.15.102:0/217390970 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58038020 con 0x56256dd3b230 2024-04-24T23:13:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.163+0000 7f0b49ffb640 1 -- 172.21.15.102:0/217390970 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b5801f5f0 con 0x56256dd3b230 2024-04-24T23:13:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.163+0000 7f0b49ffb640 1 -- 172.21.15.102:0/217390970 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58023040 con 0x56256dd3b230 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 -- 172.21.15.102:0/217390970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 msgr2=0x56256dd372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 --2- 172.21.15.102:0/217390970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd372e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0b5801fab0 tx=0x7f0b580143e0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 -- 172.21.15.102:0/217390970 shutdown_connections 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 --2- 172.21.15.102:0/217390970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 0x56256dd38850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 --2- 172.21.15.102:0/217390970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd372e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 --2- 172.21.15.102:0/217390970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd46f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 -- 172.21.15.102:0/217390970 >> 172.21.15.102:0/217390970 conn(0x56256dd2c560 msgr2=0x56256dd38140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 -- 172.21.15.102:0/217390970 shutdown_connections 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 -- 172.21.15.102:0/217390970 wait complete. 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 -- start start 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.164+0000 7f0b6d210740 1 Processor -- start 2024-04-24T23:13:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 0x56256dd43700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b6d210740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd3e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b6d210740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641ab890 con 0x56256dd3b230 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b6d210740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641acd00 con 0x56256dd30240 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b6d210740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641abf60 con 0x56256dd329a0 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b6d210740 10 client.0 _ll_register_callbacks cb 0x7f0b580096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b689c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd3e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b689c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd3e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50172/0 (socket says 172.21.15.102:50172) 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b689c9640 1 -- 172.21.15.102:0/324515268 learned_addr learned my addr 172.21.15.102:0/324515268 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b63fff640 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b637fe640 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 0x56256dd43700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b689c9640 1 -- 172.21.15.102:0/324515268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 msgr2=0x56256dd43700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b689c9640 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 0x56256dd43700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b689c9640 1 -- 172.21.15.102:0/324515268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 msgr2=0x56256dd431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:13:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b689c9640 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b689c9640 1 -- 172.21.15.102:0/324515268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256dd3ef00 con 0x56256dd3b230 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b689c9640 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd3e790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009870 tx=0x7f0b5c0307d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b93fa0d11dddae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.165+0000 7f0b437fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b437fe640 1 -- 172.21.15.102:0/324515268 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0bc5e0 con 0x56256dd3b230 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b437fe640 1 -- 172.21.15.102:0/324515268 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f0b5c0bc2d0 con 0x56256dd3b230 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b6d210740 1 -- 172.21.15.102:0/324515268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b5801f5f0 con 0x56256dd3b230 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b437fe640 1 -- 172.21.15.102:0/324515268 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0b5c0305a0 con 0x56256dd3b230 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b6d210740 10 client.15465 fetch_fsmap learned FSMap version 21 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b6d210740 1 -- 172.21.15.102:0/324515268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56256ddb6230 con 0x56256dd3b230 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b437fe640 1 -- 172.21.15.102:0/324515268 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b5801f5f0 con 0x56256dd3b230 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b437fe640 1 -- 172.21.15.102:0/324515268 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0b5c030070 con 0x56256dd3b230 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b6d210740 10 client.15465 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b6d210740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b6d210740 20 client.15465 populate_metadata read hostname 'smithi102' 2024-04-24T23:13:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.166+0000 7f0b6d210740 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56256ddb6950 0x56256ddb8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.167+0000 7f0b6d210740 4 client.15465 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:13:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.167+0000 7f0b6d210740 1 -- 172.21.15.102:0/324515268 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56256dc9c540 con 0x56256ddb6950 2024-04-24T23:13:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.167+0000 7f0b63fff640 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56256ddb6950 0x56256ddb8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.167+0000 7f0b63fff640 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56256ddb6950 0x56256ddb8d70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47010ce8102540f server_cookie=a2329ed1c9e9d09f in_seq=0 out_seq=0 2024-04-24T23:13:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.167+0000 7f0b437fe640 10 client.15465 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b437fe640 1 -- 172.21.15.102:0/324515268 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56256dc9c540 con 0x56256ddb6950 2024-04-24T23:13:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b437fe640 10 client.15465 handle_command_reply: tid=0 2024-04-24T23:13:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 client.15465 shutdown 2024-04-24T23:13:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 -- 172.21.15.102:0/324515268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 msgr2=0x56256dd3e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd3e790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009870 tx=0x7f0b5c0307d0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 -- 172.21.15.102:0/324515268 shutdown_connections 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56256ddb6950 0x56256ddb8d70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b437fe640 0 client.15465 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56256dd3b230 0x56256dd3e790 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56256dd329a0 0x56256dd43700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 --2- 172.21.15.102:0/324515268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256dd30240 0x56256dd431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 -- 172.21.15.102:0/324515268 >> 172.21.15.102:0/324515268 conn(0x56256dd2c560 msgr2=0x56256dd32220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 -- 172.21.15.102:0/324515268 shutdown_connections 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 1 -- 172.21.15.102:0/324515268 wait complete. 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.171+0000 7f0b6d210740 20 client.15465 trim_cache size 0 max 16384 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:13:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:13:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:13:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 -- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0b4c0a2f40 msgr2=0x7f0b4c0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0b4c0a2f40 0x7f0b4c0a5400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b64077ee0 tx=0x7f0b58009110 comp rx=0 tx=0).stop 2024-04-24T23:13:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 -- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64106630 msgr2=0x7f0b6419c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64106630 0x7f0b6419c810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c005e30 tx=0x7f0b5c003ba0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 -- 172.21.15.102:0/3323461138 shutdown_connections 2024-04-24T23:13:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0b4c0a2f40 0x7f0b4c0a5400 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b64106630 0x7f0b6419c810 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b64101030 0x7f0b64198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 --2- 172.21.15.102:0/3323461138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b64100670 0x7f0b64197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 -- 172.21.15.102:0/3323461138 >> 172.21.15.102:0/3323461138 conn(0x7f0b640777c0 msgr2=0x7f0b64069130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 -- 172.21.15.102:0/3323461138 shutdown_connections 2024-04-24T23:13:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:13:57.172+0000 7f0b6a42c640 1 -- 172.21.15.102:0/3323461138 wait complete. 2024-04-24T23:13:57.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:57 smithi183 ceph-mon[33137]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:57.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:57 smithi102 ceph-mon[29216]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:57 smithi161 ceph-mon[32709]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:13:59.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:13:59 smithi102 ceph-mon[29216]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-24T23:13:59.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:13:59 smithi161 ceph-mon[32709]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-24T23:13:59.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:13:59 smithi183 ceph-mon[33137]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-24T23:14:01.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:14:00 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:14:00.813+0000 7f8b30423700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:14:01.072 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:14:00 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:14:00.814+0000 7f8b30423700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:14:01.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:14:00 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:14:00.814+0000 7f8b30423700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:14:01.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:14:00 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:14:00.814+0000 7f8b30423700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:14:01.073 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:14:00 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:14:00.814+0000 7f8b30423700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:14:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:01 smithi102 ceph-mon[29216]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:14:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:01 smithi161 ceph-mon[32709]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:14:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:01 smithi183 ceph-mon[33137]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:14:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:03 smithi102 ceph-mon[29216]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:14:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:03 smithi102 ceph-mon[29216]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-04-24T23:14:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:03 smithi161 ceph-mon[32709]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:14:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:03 smithi161 ceph-mon[32709]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-04-24T23:14:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:03 smithi183 ceph-mon[33137]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T23:14:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:03 smithi183 ceph-mon[33137]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-04-24T23:14:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:05 smithi102 ceph-mon[29216]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T23:14:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:05 smithi161 ceph-mon[32709]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T23:14:05.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:05 smithi183 ceph-mon[33137]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T23:14:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:07 smithi102 ceph-mon[29216]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-24T23:14:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:07 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:07 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:14:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:07 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:14:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:07 smithi161 ceph-mon[32709]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-24T23:14:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:07 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:07 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:14:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:07 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:14:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:07 smithi183 ceph-mon[33137]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-24T23:14:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:07 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:07 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:14:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:07 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:14:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:08 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:08 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:14:08.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:08 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:14:08.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:08 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:08.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:08 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:14:08.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:08 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:14:08.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:08 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:08.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:08 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:14:08.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:08 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:14:09.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:09 smithi102 ceph-mon[29216]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T23:14:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:09 smithi161 ceph-mon[32709]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T23:14:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:09 smithi183 ceph-mon[33137]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T23:14:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.303+0000 7f9dfc464640 1 -- 172.21.15.102:0/38504332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df4100000 msgr2=0x7f9df41003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.303+0000 7f9dfc464640 1 --2- 172.21.15.102:0/38504332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df4100000 0x7f9df41003e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9df000aa10 tx=0x7f9df002b720 comp rx=0 tx=0).stop 2024-04-24T23:14:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.303+0000 7f9dfc464640 1 -- 172.21.15.102:0/38504332 shutdown_connections 2024-04-24T23:14:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.303+0000 7f9dfc464640 1 --2- 172.21.15.102:0/38504332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df41050d0 0x7f9df410cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.303+0000 7f9dfc464640 1 --2- 172.21.15.102:0/38504332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df41009b0 0x7f9df41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.303+0000 7f9dfc464640 1 --2- 172.21.15.102:0/38504332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df4100000 0x7f9df41003e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.303+0000 7f9dfc464640 1 -- 172.21.15.102:0/38504332 >> 172.21.15.102:0/38504332 conn(0x7f9df4076bd0 msgr2=0x7f9df40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.303+0000 7f9dfc464640 1 -- 172.21.15.102:0/38504332 shutdown_connections 2024-04-24T23:14:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.303+0000 7f9dfc464640 1 -- 172.21.15.102:0/38504332 wait complete. 2024-04-24T23:14:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.304+0000 7f9dfc464640 1 Processor -- start 2024-04-24T23:14:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.304+0000 7f9dfc464640 1 -- start start 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.304+0000 7f9dfc464640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df4100000 0x7f9df419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfc464640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df41050d0 0x7f9df419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfc464640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df41958a0 0x7f9df4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfc464640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9df410f800 con 0x7f9df41958a0 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfc464640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9df410f680 con 0x7f9df4100000 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfc464640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9df410f980 con 0x7f9df41050d0 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfaa01640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df41958a0 0x7f9df4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfaa01640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df41958a0 0x7f9df4195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33678/0 (socket says 172.21.15.102:33678) 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfaa01640 1 -- 172.21.15.102:0/3550075216 learned_addr learned my addr 172.21.15.102:0/3550075216 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfa200640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df4100000 0x7f9df419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df41050d0 0x7f9df419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfaa01640 1 -- 172.21.15.102:0/3550075216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df41050d0 msgr2=0x7f9df419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfaa01640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df41050d0 0x7f9df419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfaa01640 1 -- 172.21.15.102:0/3550075216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df4100000 msgr2=0x7f9df419b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfaa01640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df4100000 0x7f9df419b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfaa01640 1 -- 172.21.15.102:0/3550075216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df41963c0 con 0x7f9df41958a0 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df41050d0 0x7f9df419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfa200640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df4100000 0x7f9df419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.305+0000 7f9dfaa01640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df41958a0 0x7f9df4195d50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00cce0 tx=0x7f9dec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.306+0000 7f9deb7fe640 1 -- 172.21.15.102:0/3550075216 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec00f450 con 0x7f9df41958a0 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.306+0000 7f9deb7fe640 1 -- 172.21.15.102:0/3550075216 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dec00faa0 con 0x7f9df41958a0 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.306+0000 7f9dfc464640 1 -- 172.21.15.102:0/3550075216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df41966b0 con 0x7f9df41958a0 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.306+0000 7f9deb7fe640 1 -- 172.21.15.102:0/3550075216 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec018b70 con 0x7f9df41958a0 2024-04-24T23:14:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.306+0000 7f9dfc464640 1 -- 172.21.15.102:0/3550075216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df41a2900 con 0x7f9df41958a0 2024-04-24T23:14:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.309+0000 7f9deb7fe640 1 -- 172.21.15.102:0/3550075216 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 25) ==== 138118+0+0 (secure 0 0 0) 0x7f9dec018d10 con 0x7f9df41958a0 2024-04-24T23:14:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9dff172740 1 Processor -- start 2024-04-24T23:14:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9dff172740 1 -- start start 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9dff172740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181eef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9dff172740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 0x565181efe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9deb7fe640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9de00a3050 0x7f9de00a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9dff172740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f00c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9dff172740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x565181e53cb0 con 0x565181efe830 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9dff172740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565181dfc5b0 con 0x565181eeeef0 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9dff172740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565181e4fc70 con 0x565181eef810 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9df99ff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181eef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9df99ff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181eef2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:52760/0 (socket says 172.21.15.102:52760) 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.310+0000 7f9df99ff640 1 -- 172.21.15.102:0/3437518110 learned_addr learned my addr 172.21.15.102:0/3437518110 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9deb7fe640 1 -- 172.21.15.102:0/3550075216 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9dec0cfab0 con 0x7f9df41958a0 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9dfaa01640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9de00a3050 0x7f9de00a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3437518110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f00c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9dfa200640 1 --2- 172.21.15.102:0/3437518110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 0x565181efe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9df99ff640 1 -- 172.21.15.102:0/3437518110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 msgr2=0x565181efe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3437518110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 0x565181efe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9df99ff640 1 -- 172.21.15.102:0/3437518110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 msgr2=0x565181eef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3437518110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181eef2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9df99ff640 1 -- 172.21.15.102:0/3437518110 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565181b68210 con 0x565181efe830 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9dfaa01640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9de00a3050 0x7f9de00a5510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0cd010 tx=0x7f9dec0185b0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.311+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3437518110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f00c20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009ca0 tx=0x7f9de4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f06c069ec74801 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.312+0000 7f9de8bfe640 1 -- 172.21.15.102:0/3437518110 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de401f050 con 0x565181efe830 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.312+0000 7f9de8bfe640 1 -- 172.21.15.102:0/3437518110 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9de401d070 con 0x565181efe830 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.312+0000 7f9de8bfe640 1 -- 172.21.15.102:0/3437518110 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4003850 con 0x565181efe830 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.312+0000 7f9dff172740 1 -- 172.21.15.102:0/3437518110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 msgr2=0x565181f00c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.312+0000 7f9dff172740 1 --2- 172.21.15.102:0/3437518110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f00c20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009ca0 tx=0x7f9de4004730 comp rx=0 tx=0).stop 2024-04-24T23:14:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 -- 172.21.15.102:0/3437518110 shutdown_connections 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 --2- 172.21.15.102:0/3437518110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f00c20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 --2- 172.21.15.102:0/3437518110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 0x565181efe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 --2- 172.21.15.102:0/3437518110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181eef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 -- 172.21.15.102:0/3437518110 >> 172.21.15.102:0/3437518110 conn(0x565181eeb560 msgr2=0x565181eeb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 -- 172.21.15.102:0/3437518110 shutdown_connections 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 -- 172.21.15.102:0/3437518110 wait complete. 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 -- start start 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 Processor -- start 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181ef7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eef810 0x565181ef7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181efe830 0x565181efc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x565181ef40e0 con 0x565181eef810 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565181e0fb50 con 0x565181eeeef0 2024-04-24T23:14:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565181ef4260 con 0x565181efe830 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dff172740 10 client.0 _ll_register_callbacks cb 0x565181e4aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dfa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eef810 0x565181ef7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dfa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eef810 0x565181ef7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33688/0 (socket says 172.21.15.102:33688) 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.313+0000 7f9dfa200640 1 -- 172.21.15.102:0/76000271 learned_addr learned my addr 172.21.15.102:0/76000271 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9df99ff640 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181ef7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dfa200640 1 -- 172.21.15.102:0/76000271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181efe830 msgr2=0x565181efc230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dfa200640 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181efe830 0x565181efc230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dfa200640 1 -- 172.21.15.102:0/76000271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 msgr2=0x565181ef7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dfa200640 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181ef7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dfa200640 1 -- 172.21.15.102:0/76000271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565181c34620 con 0x565181eef810 2024-04-24T23:14:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9df99ff640 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181ef7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dfa200640 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eef810 0x565181ef7fb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9df000aa10 tx=0x7f9df0030b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dca3f601affbf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dddffb640 1 -- 172.21.15.102:0/76000271 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0036070 con 0x565181eef810 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dddffb640 1 -- 172.21.15.102:0/76000271 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9df00037f0 con 0x565181eef810 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dff172740 1 -- 172.21.15.102:0/76000271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9de401d070 con 0x565181eef810 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dddffb640 1 -- 172.21.15.102:0/76000271 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9df00039f0 con 0x565181eef810 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dff172740 10 client.15483 fetch_fsmap learned FSMap version 21 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.314+0000 7f9dff172740 1 -- 172.21.15.102:0/76000271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x565181f092d0 con 0x565181eef810 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.315+0000 7f9dddffb640 1 -- 172.21.15.102:0/76000271 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9de401d070 con 0x565181eef810 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.315+0000 7f9dddffb640 1 -- 172.21.15.102:0/76000271 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9df0044020 con 0x565181eef810 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.315+0000 7f9dff172740 10 client.15483 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.315+0000 7f9dff172740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.315+0000 7f9dff172740 20 client.15483 populate_metadata read hostname 'smithi102' 2024-04-24T23:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.315+0000 7f9dff172740 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565181f0a7e0 0x565181f0cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.315+0000 7f9dff172740 4 client.15483 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.315+0000 7f9dff172740 1 -- 172.21.15.102:0/76000271 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565181e611f0 con 0x565181f0a7e0 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.315+0000 7f9df99ff640 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565181f0a7e0 0x565181f0cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.316+0000 7f9dddffb640 10 client.15483 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.316+0000 7f9df99ff640 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565181f0a7e0 0x565181f0cc50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737ed3eec91e4ecd server_cookie=d11dd83eb84e337 in_seq=0 out_seq=0 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.316+0000 7f9dddffb640 1 -- 172.21.15.102:0/76000271 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565181e611f0 con 0x565181f0a7e0 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.316+0000 7f9dddffb640 10 client.15483 handle_command_reply: tid=0 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.316+0000 7f9dff172740 1 client.15483 shutdown 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 -- 172.21.15.102:0/76000271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eef810 msgr2=0x565181ef7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eef810 0x565181ef7fb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9df000aa10 tx=0x7f9df0030b90 comp rx=0 tx=0).stop 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 -- 172.21.15.102:0/76000271 shutdown_connections 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181efe830 0x565181efc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565181f0a7e0 0x565181f0cc50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eef810 0x565181ef7fb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 --2- 172.21.15.102:0/76000271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181ef7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 -- 172.21.15.102:0/76000271 >> 172.21.15.102:0/76000271 conn(0x565181eeb560 msgr2=0x565181ef12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dddffb640 0 client.15483 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 -- 172.21.15.102:0/76000271 shutdown_connections 2024-04-24T23:14:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 1 -- 172.21.15.102:0/76000271 wait complete. 2024-04-24T23:14:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.317+0000 7f9dff172740 20 client.15483 trim_cache size 0 max 16384 2024-04-24T23:14:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.331+0000 7f9dff172740 1 Processor -- start 2024-04-24T23:14:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.331+0000 7f9dff172740 1 -- start start 2024-04-24T23:14:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9dff172740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eeeef0 0x565181efc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9dff172740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eef810 0x565181ef5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9dff172740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181efe830 0x565181eeacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9dff172740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41ad780 con 0x565181eeeef0 2024-04-24T23:14:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9dff172740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41ab050 con 0x565181eef810 2024-04-24T23:14:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9dff172740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41a8ec0 con 0x565181efe830 2024-04-24T23:14:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9df99ff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eeeef0 0x565181efc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9df99ff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eeeef0 0x565181efc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33694/0 (socket says 172.21.15.102:33694) 2024-04-24T23:14:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9df99ff640 1 -- 172.21.15.102:0/3173633437 learned_addr learned my addr 172.21.15.102:0/3173633437 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9dfa200640 1 --2- 172.21.15.102:0/3173633437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eef810 0x565181ef5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9df99ff640 1 -- 172.21.15.102:0/3173633437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181efe830 msgr2=0x565181eeacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3173633437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181efe830 0x565181eeacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9df99ff640 1 -- 172.21.15.102:0/3173633437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eef810 msgr2=0x565181ef5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3173633437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eef810 0x565181ef5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9df99ff640 1 -- 172.21.15.102:0/3173633437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565181f0cfc0 con 0x565181eeeef0 2024-04-24T23:14:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9dfa200640 1 --2- 172.21.15.102:0/3173633437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eef810 0x565181ef5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.332+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3173633437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eeeef0 0x565181efc090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x565181eec6f0 tx=0x7f9de4018ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36842961e361d78c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9ddffff640 1 -- 172.21.15.102:0/3173633437 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de401f040 con 0x565181eeeef0 2024-04-24T23:14:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9ddffff640 1 -- 172.21.15.102:0/3173633437 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9de401d040 con 0x565181eeeef0 2024-04-24T23:14:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9ddffff640 1 -- 172.21.15.102:0/3173633437 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4030670 con 0x565181eeeef0 2024-04-24T23:14:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9dff172740 1 -- 172.21.15.102:0/3173633437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eeeef0 msgr2=0x565181efc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9dff172740 1 --2- 172.21.15.102:0/3173633437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eeeef0 0x565181efc090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x565181eec6f0 tx=0x7f9de4018ed0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9dff172740 1 -- 172.21.15.102:0/3173633437 shutdown_connections 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9dff172740 1 --2- 172.21.15.102:0/3173633437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181efe830 0x565181eeacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9dff172740 1 --2- 172.21.15.102:0/3173633437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eef810 0x565181ef5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9dff172740 1 --2- 172.21.15.102:0/3173633437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181eeeef0 0x565181efc090 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9dff172740 1 -- 172.21.15.102:0/3173633437 >> 172.21.15.102:0/3173633437 conn(0x565181eeb560 msgr2=0x565181eff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9dff172740 1 -- 172.21.15.102:0/3173633437 shutdown_connections 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.333+0000 7f9dff172740 1 -- 172.21.15.102:0/3173633437 wait complete. 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9dff172740 1 -- start start 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9dff172740 1 Processor -- start 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9dff172740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181f11550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9dff172740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 0x565181f11a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9dff172740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f11fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9dff172740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41a9510 con 0x565181efe830 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9dff172740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41aa980 con 0x565181eeeef0 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9dff172740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41a9be0 con 0x565181eef810 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9dff172740 10 client.0 _ll_register_callbacks cb 0x7f9df000a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9df99ff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f11fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9df99ff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f11fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33710/0 (socket says 172.21.15.102:33710) 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.334+0000 7f9df99ff640 1 -- 172.21.15.102:0/3015235479 learned_addr learned my addr 172.21.15.102:0/3015235479 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9dfa200640 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 0x565181f11a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9df99ff640 1 -- 172.21.15.102:0/3015235479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 msgr2=0x565181f11a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 0x565181f11a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9df99ff640 1 -- 172.21.15.102:0/3015235479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 msgr2=0x565181f11550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181f11550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9df99ff640 1 -- 172.21.15.102:0/3015235479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565181f6f010 con 0x565181efe830 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9df99ff640 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f11fd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005d60 tx=0x7f9de40095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e926e3b54bc7329c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9dddffb640 1 -- 172.21.15.102:0/3015235479 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de401f040 con 0x565181efe830 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9dff172740 1 -- 172.21.15.102:0/3015235479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9de401d040 con 0x565181efe830 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9dddffb640 1 -- 172.21.15.102:0/3015235479 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9de4030410 con 0x565181efe830 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9dddffb640 1 -- 172.21.15.102:0/3015235479 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9de4004510 con 0x565181efe830 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9dff172740 10 client.15489 fetch_fsmap learned FSMap version 21 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.335+0000 7f9dff172740 1 -- 172.21.15.102:0/3015235479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x565181f702e0 con 0x565181efe830 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.336+0000 7f9dddffb640 1 -- 172.21.15.102:0/3015235479 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9de401d040 con 0x565181efe830 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.336+0000 7f9dddffb640 1 -- 172.21.15.102:0/3015235479 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9de40050b0 con 0x565181efe830 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.336+0000 7f9dff172740 10 client.15489 fetch_fsmap finished waiting for FSMap version 21 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.336+0000 7f9dff172740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.336+0000 7f9dff172740 20 client.15489 populate_metadata read hostname 'smithi102' 2024-04-24T23:14:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.336+0000 7f9dff172740 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565181f707f0 0x565181f72c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.336+0000 7f9dff172740 4 client.15489 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:14:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.336+0000 7f9dff172740 1 -- 172.21.15.102:0/3015235479 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565181e5b540 con 0x565181f707f0 2024-04-24T23:14:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.336+0000 7f9dfa200640 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565181f707f0 0x565181f72c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.337+0000 7f9dfa200640 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565181f707f0 0x565181f72c10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=766f5652b55d2eb9 server_cookie=689172a1a7b57ab0 in_seq=0 out_seq=0 2024-04-24T23:14:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.337+0000 7f9dddffb640 10 client.15489 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dddffb640 1 -- 172.21.15.102:0/3015235479 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565181e5b540 con 0x565181f707f0 2024-04-24T23:14:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dddffb640 10 client.15489 handle_command_reply: tid=0 2024-04-24T23:14:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 client.15489 shutdown 2024-04-24T23:14:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 -- 172.21.15.102:0/3015235479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 msgr2=0x565181f11fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f11fd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005d60 tx=0x7f9de40095b0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 -- 172.21.15.102:0/3015235479 shutdown_connections 2024-04-24T23:14:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565181f707f0 0x565181f72c10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dddffb640 0 client.15489 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565181efe830 0x565181f11fd0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565181eef810 0x565181f11a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 --2- 172.21.15.102:0/3015235479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565181eeeef0 0x565181f11550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 -- 172.21.15.102:0/3015235479 >> 172.21.15.102:0/3015235479 conn(0x565181eeb560 msgr2=0x565181ef6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 -- 172.21.15.102:0/3015235479 shutdown_connections 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 1 -- 172.21.15.102:0/3015235479 wait complete. 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.343+0000 7f9dff172740 20 client.15489 trim_cache size 0 max 16384 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:14:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:14:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 -- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9de00a3050 msgr2=0x7f9de00a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9de00a3050 0x7f9de00a5510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0cd010 tx=0x7f9dec0185b0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 -- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df41958a0 msgr2=0x7f9df4195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df41958a0 0x7f9df4195d50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00cce0 tx=0x7f9dec00a620 comp rx=0 tx=0).stop 2024-04-24T23:14:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 -- 172.21.15.102:0/3550075216 shutdown_connections 2024-04-24T23:14:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9de00a3050 0x7f9de00a5510 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9df41958a0 0x7f9df4195d50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df41050d0 0x7f9df419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 --2- 172.21.15.102:0/3550075216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df4100000 0x7f9df419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 -- 172.21.15.102:0/3550075216 >> 172.21.15.102:0/3550075216 conn(0x7f9df4076bd0 msgr2=0x7f9df4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 -- 172.21.15.102:0/3550075216 shutdown_connections 2024-04-24T23:14:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:10.344+0000 7f9dfc464640 1 -- 172.21.15.102:0/3550075216 wait complete. 2024-04-24T23:14:11.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:11 smithi102 ceph-mon[29216]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T23:14:11.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:11 smithi161 ceph-mon[32709]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T23:14:11.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:11 smithi183 ceph-mon[33137]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T23:14:13.314 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:13 smithi102 ceph-mon[29216]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T23:14:13.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:13 smithi161 ceph-mon[32709]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T23:14:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:13 smithi183 ceph-mon[33137]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T23:14:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:15 smithi102 ceph-mon[29216]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T23:14:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:15 smithi161 ceph-mon[32709]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T23:14:15.763 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:15 smithi183 ceph-mon[33137]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T23:14:16.278 INFO:tasks.workunit.client.0.smithi102.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T23:14:16.278 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-24T23:14:16.278 INFO:tasks.workunit.client.0.smithi102.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T23:14:16.278 INFO:tasks.workunit.client.0.smithi102.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr: git switch -c 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr:Or undo this operation with: 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr: git switch - 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-04-24T23:14:16.279 INFO:tasks.workunit.client.0.smithi102.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T23:14:16.281 DEBUG:teuthology.orchestra.run.smithi102:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T23:14:16.359 INFO:tasks.workunit.client.0.smithi102.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T23:14:16.361 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:14:16.361 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T23:14:17.009 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T23:14:17.062 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T23:14:17.101 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:14:17.102 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:14:17.102 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T23:14:17.140 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:14:17.144 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:14:17.144 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T23:14:17.200 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-04-24T23:14:17.202 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-04-24T23:14:17.202 DEBUG:teuthology.orchestra.run.smithi102:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-24T23:14:17.266 INFO:tasks.workunit.client.0.smithi102.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-24T23:14:17.266 INFO:tasks.workunit.client.0.smithi102.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-04-24T23:14:17.266 INFO:tasks.workunit.client.0.smithi102.stderr:+ trap -- retry EXIT 2024-04-24T23:14:17.267 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm -rf ffsb 2024-04-24T23:14:17.267 INFO:tasks.workunit.client.0.smithi102.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-04-24T23:14:17.271 INFO:tasks.workunit.client.0.smithi102.stderr:Cloning into 'ffsb'... 2024-04-24T23:14:17.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:17 smithi102 ceph-mon[29216]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T23:14:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:17 smithi161 ceph-mon[32709]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T23:14:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:17 smithi183 ceph-mon[33137]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T23:14:18.982 INFO:tasks.workunit.client.0.smithi102.stderr:+ trap - EXIT 2024-04-24T23:14:18.983 INFO:tasks.workunit.client.0.smithi102.stderr:+ cd ffsb 2024-04-24T23:14:18.983 INFO:tasks.workunit.client.0.smithi102.stderr:+ ./configure 2024-04-24T23:14:19.111 INFO:tasks.workunit.client.0.smithi102.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-24T23:14:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:19 smithi102 ceph-mon[29216]: pgmap v41: 97 pgs: 97 active+clean; 1.5 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-04-24T23:14:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:19 smithi161 ceph-mon[32709]: pgmap v41: 97 pgs: 97 active+clean; 1.5 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-04-24T23:14:19.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:19 smithi183 ceph-mon[33137]: pgmap v41: 97 pgs: 97 active+clean; 1.5 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-04-24T23:14:20.122 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether build environment is sane... yes 2024-04-24T23:14:20.138 INFO:tasks.workunit.client.0.smithi102.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-24T23:14:20.139 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gawk... gawk 2024-04-24T23:14:20.169 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether make sets $(MAKE)... yes 2024-04-24T23:14:20.215 INFO:tasks.workunit.client.0.smithi102.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-24T23:14:20.216 INFO:tasks.workunit.client.0.smithi102.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-24T23:14:20.239 INFO:tasks.workunit.client.0.smithi102.stdout:checking for style of include used by make... GNU 2024-04-24T23:14:20.244 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gcc... gcc 2024-04-24T23:14:20.329 INFO:tasks.workunit.client.0.smithi102.stdout:checking for C compiler default output file name... a.out 2024-04-24T23:14:20.331 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether the C compiler works... yes 2024-04-24T23:14:20.335 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether we are cross compiling... no 2024-04-24T23:14:20.385 INFO:tasks.workunit.client.0.smithi102.stdout:checking for suffix of executables... 2024-04-24T23:14:20.427 INFO:tasks.workunit.client.0.smithi102.stdout:checking for suffix of object files... o 2024-04-24T23:14:20.469 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether we are using the GNU C compiler... yes 2024-04-24T23:14:20.510 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether gcc accepts -g... yes 2024-04-24T23:14:20.594 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-24T23:14:20.751 INFO:tasks.workunit.client.0.smithi102.stdout:checking dependency style of gcc... gcc3 2024-04-24T23:14:20.840 INFO:tasks.workunit.client.0.smithi102.stdout:checking how to run the C preprocessor... gcc -E 2024-04-24T23:14:20.897 INFO:tasks.workunit.client.0.smithi102.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-24T23:14:20.898 INFO:tasks.workunit.client.0.smithi102.stdout:checking for egrep... /usr/bin/grep -E 2024-04-24T23:14:20.911 INFO:tasks.workunit.client.0.smithi102.stdout:checking for AIX... no 2024-04-24T23:14:20.940 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gcc... (cached) gcc 2024-04-24T23:14:20.951 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-24T23:14:20.952 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-24T23:14:20.953 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-24T23:14:20.953 INFO:tasks.workunit.client.0.smithi102.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-24T23:14:21.026 INFO:tasks.workunit.client.0.smithi102.stdout:checking for main in -lm... yes 2024-04-24T23:14:21.101 INFO:tasks.workunit.client.0.smithi102.stdout:checking for main in -lpthread... yes 2024-04-24T23:14:21.303 INFO:tasks.workunit.client.0.smithi102.stdout:checking for ANSI C header files... yes 2024-04-24T23:14:21.489 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-24T23:14:21.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:21 smithi161 ceph-mon[32709]: pgmap v42: 97 pgs: 97 active+clean; 1.5 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-24T23:14:21.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:21 smithi183 ceph-mon[33137]: pgmap v42: 97 pgs: 97 active+clean; 1.5 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-24T23:14:21.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:21 smithi102 ceph-mon[29216]: pgmap v42: 97 pgs: 97 active+clean; 1.5 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-24T23:14:22.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:22 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:14:22.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:22 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:14:22.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:22 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 -- 172.21.15.102:0/3412369057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 msgr2=0x7fb95010d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 --2- 172.21.15.102:0/3412369057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 0x7fb95010d4f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb948009cd0 tx=0x7fb94802a8e0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 -- 172.21.15.102:0/3412369057 shutdown_connections 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 --2- 172.21.15.102:0/3412369057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 0x7fb95010d4f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 --2- 172.21.15.102:0/3412369057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb950100670 0x7fb950100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 --2- 172.21.15.102:0/3412369057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb950106630 0x7fb950106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 -- 172.21.15.102:0/3412369057 >> 172.21.15.102:0/3412369057 conn(0x7fb9500777c0 msgr2=0x7fb950077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 -- 172.21.15.102:0/3412369057 shutdown_connections 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 -- 172.21.15.102:0/3412369057 wait complete. 2024-04-24T23:14:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 Processor -- start 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 -- start start 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb950100670 0x7fb950197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 0x7fb950198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb950106630 0x7fb95019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501108f0 con 0x7fb950101030 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb950110770 con 0x7fb950100670 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.443+0000 7fb956c6b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb950110a70 con 0x7fb950106630 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 0x7fb950198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 0x7fb950198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60596/0 (socket says 172.21.15.102:60596) 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94ffff640 1 -- 172.21.15.102:0/920769852 learned_addr learned my addr 172.21.15.102:0/920769852 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb955208640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb950106630 0x7fb95019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb954a07640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb950100670 0x7fb950197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94ffff640 1 -- 172.21.15.102:0/920769852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb950106630 msgr2=0x7fb95019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94ffff640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb950106630 0x7fb95019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94ffff640 1 -- 172.21.15.102:0/920769852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb950100670 msgr2=0x7fb950197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94ffff640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb950100670 0x7fb950197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94ffff640 1 -- 172.21.15.102:0/920769852 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95019ce00 con 0x7fb950101030 2024-04-24T23:14:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94ffff640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 0x7fb950198350 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00d950 tx=0x7fb93c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94dffb640 1 -- 172.21.15.102:0/920769852 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c017070 con 0x7fb950101030 2024-04-24T23:14:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94dffb640 1 -- 172.21.15.102:0/920769852 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb93c009c30 con 0x7fb950101030 2024-04-24T23:14:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb956c6b640 1 -- 172.21.15.102:0/920769852 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95019d0f0 con 0x7fb950101030 2024-04-24T23:14:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb94dffb640 1 -- 172.21.15.102:0/920769852 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00f620 con 0x7fb950101030 2024-04-24T23:14:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.444+0000 7fb956c6b640 1 -- 172.21.15.102:0/920769852 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9501a4980 con 0x7fb950101030 2024-04-24T23:14:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.446+0000 7fb94dffb640 1 -- 172.21.15.102:0/920769852 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 25) ==== 138118+0+0 (secure 0 0 0) 0x7fb93c0109a0 con 0x7fb950101030 2024-04-24T23:14:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb95997b740 1 Processor -- start 2024-04-24T23:14:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb95997b740 1 -- start start 2024-04-24T23:14:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb94dffb640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9400a2f50 0x7fb9400a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb95997b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046240 0x5556a2046620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb95997b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 0x5556a2050cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb95997b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2051230 0x5556a2053620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb95997b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5556a1faacb0 con 0x5556a2046240 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb95997b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5556a1f535b0 con 0x5556a2051230 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb95997b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5556a1fa6c70 con 0x5556a2046b60 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb955208640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9400a2f50 0x7fb9400a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb954a07640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046240 0x5556a2046620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb954a07640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046240 0x5556a2046620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60610/0 (socket says 172.21.15.102:60610) 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb954a07640 1 -- 172.21.15.102:0/3427016300 learned_addr learned my addr 172.21.15.102:0/3427016300 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.447+0000 7fb94dffb640 1 -- 172.21.15.102:0/920769852 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb93c0cf500 con 0x7fb950101030 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb94ffff640 1 --2- 172.21.15.102:0/3427016300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 0x5556a2050cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb955208640 1 --2- 172.21.15.102:0/3427016300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2051230 0x5556a2053620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb954a07640 1 -- 172.21.15.102:0/3427016300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 msgr2=0x5556a2050cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb954a07640 1 --2- 172.21.15.102:0/3427016300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 0x5556a2050cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb954a07640 1 -- 172.21.15.102:0/3427016300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2051230 msgr2=0x5556a2053620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb954a07640 1 --2- 172.21.15.102:0/3427016300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2051230 0x5556a2053620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb954a07640 1 -- 172.21.15.102:0/3427016300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a1cbf210 con 0x5556a2046240 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb955208640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9400a2f50 0x7fb9400a5410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb948009ca0 tx=0x7fb948031630 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb954a07640 1 --2- 172.21.15.102:0/3427016300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046240 0x5556a2046620 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb94400ba30 tx=0x7fb94400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950296f9e603f6a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb93affd640 1 -- 172.21.15.102:0/3427016300 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944017420 con 0x5556a2046240 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb93affd640 1 -- 172.21.15.102:0/3427016300 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb944017a70 con 0x5556a2046240 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.448+0000 7fb93affd640 1 -- 172.21.15.102:0/3427016300 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94400f970 con 0x5556a2046240 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 -- 172.21.15.102:0/3427016300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046240 msgr2=0x5556a2046620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 --2- 172.21.15.102:0/3427016300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046240 0x5556a2046620 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb94400ba30 tx=0x7fb94400bef0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 -- 172.21.15.102:0/3427016300 shutdown_connections 2024-04-24T23:14:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 --2- 172.21.15.102:0/3427016300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2051230 0x5556a2053620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 --2- 172.21.15.102:0/3427016300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 0x5556a2050cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 --2- 172.21.15.102:0/3427016300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046240 0x5556a2046620 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 -- 172.21.15.102:0/3427016300 >> 172.21.15.102:0/3427016300 conn(0x5556a2042560 msgr2=0x5556a2042940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 -- 172.21.15.102:0/3427016300 shutdown_connections 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 -- 172.21.15.102:0/3427016300 wait complete. 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 -- start start 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.449+0000 7fb95997b740 1 Processor -- start 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb95997b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a204e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb95997b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 0x5556a204e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb95997b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2051230 0x5556a204ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb95997b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5556a2054020 con 0x5556a2051230 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb95997b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5556a1f66b50 con 0x5556a2046240 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb95997b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5556a20541a0 con 0x5556a2046b60 2024-04-24T23:14:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb955208640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2051230 0x5556a204ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb955208640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2051230 0x5556a204ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60620/0 (socket says 172.21.15.102:60620) 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb95997b740 10 client.0 _ll_register_callbacks cb 0x5556a1fa1ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb955208640 1 -- 172.21.15.102:0/418868438 learned_addr learned my addr 172.21.15.102:0/418868438 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb94ffff640 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 0x5556a204e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb954a07640 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a204e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb955208640 1 -- 172.21.15.102:0/418868438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 msgr2=0x5556a204e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb955208640 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 0x5556a204e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb955208640 1 -- 172.21.15.102:0/418868438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 msgr2=0x5556a204e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb955208640 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a204e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb955208640 1 -- 172.21.15.102:0/418868438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a1d8b620 con 0x5556a2051230 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb954a07640 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a204e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb955208640 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2051230 0x5556a204ebd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb948009580 tx=0x7fb94802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b36d722ae09b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb92ffff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:14:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb92ffff640 1 -- 172.21.15.102:0/418868438 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94804e070 con 0x5556a2051230 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb95997b740 1 -- 172.21.15.102:0/418868438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb944017a70 con 0x5556a2051230 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.450+0000 7fb92ffff640 1 -- 172.21.15.102:0/418868438 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb9480305a0 con 0x5556a2051230 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb92ffff640 1 -- 172.21.15.102:0/418868438 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb94805a050 con 0x5556a2051230 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb95997b740 10 client.15501 fetch_fsmap learned FSMap version 23 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb95997b740 1 -- 172.21.15.102:0/418868438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5556a2060230 con 0x5556a2051230 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb92ffff640 1 -- 172.21.15.102:0/418868438 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb944017a70 con 0x5556a2051230 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb92ffff640 1 -- 172.21.15.102:0/418868438 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7fb948049070 con 0x5556a2051230 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb95997b740 10 client.15501 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb95997b740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb95997b740 20 client.15501 populate_metadata read hostname 'smithi102' 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb95997b740 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5556a2061760 0x5556a2063b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb95997b740 4 client.15501 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb95997b740 1 -- 172.21.15.102:0/418868438 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556a1fb81f0 con 0x5556a2061760 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.451+0000 7fb954a07640 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5556a2061760 0x5556a2063b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb92ffff640 10 client.15501 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb954a07640 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5556a2061760 0x5556a2063b90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c72930a9a5de37c server_cookie=1e8f8025440e9c57 in_seq=0 out_seq=0 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb92ffff640 1 -- 172.21.15.102:0/418868438 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556a1fb81f0 con 0x5556a2061760 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb92ffff640 10 client.15501 handle_command_reply: tid=0 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb95997b740 1 client.15501 shutdown 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb95997b740 1 -- 172.21.15.102:0/418868438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2051230 msgr2=0x5556a204ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb95997b740 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2051230 0x5556a204ebd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb948009580 tx=0x7fb94802a8e0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb95997b740 1 -- 172.21.15.102:0/418868438 shutdown_connections 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb95997b740 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5556a2061760 0x5556a2063b90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb95997b740 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2051230 0x5556a204ebd0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb92ffff640 0 client.15501 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb95997b740 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046b60 0x5556a204e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb95997b740 1 --2- 172.21.15.102:0/418868438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a204e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.452+0000 7fb95997b740 1 -- 172.21.15.102:0/418868438 >> 172.21.15.102:0/418868438 conn(0x5556a2042560 msgr2=0x5556a2048240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.453+0000 7fb95997b740 1 -- 172.21.15.102:0/418868438 shutdown_connections 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.453+0000 7fb95997b740 1 -- 172.21.15.102:0/418868438 wait complete. 2024-04-24T23:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.453+0000 7fb95997b740 20 client.15501 trim_cache size 0 max 16384 2024-04-24T23:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.467+0000 7fb95997b740 1 Processor -- start 2024-04-24T23:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.467+0000 7fb95997b740 1 -- start start 2024-04-24T23:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.467+0000 7fb95997b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046240 0x5556a205cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.467+0000 7fb95997b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a20489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.467+0000 7fb95997b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2051230 0x5556a204d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.467+0000 7fb95997b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501afc60 con 0x5556a2046b60 2024-04-24T23:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.467+0000 7fb95997b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ad500 con 0x5556a2051230 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.467+0000 7fb95997b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ab370 con 0x5556a2046240 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a20489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a20489a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60636/0 (socket says 172.21.15.102:60636) 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb94ffff640 1 -- 172.21.15.102:0/637190562 learned_addr learned my addr 172.21.15.102:0/637190562 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb954a07640 1 --2- 172.21.15.102:0/637190562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046240 0x5556a205cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb94ffff640 1 -- 172.21.15.102:0/637190562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046240 msgr2=0x5556a205cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb94ffff640 1 --2- 172.21.15.102:0/637190562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046240 0x5556a205cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb94ffff640 1 -- 172.21.15.102:0/637190562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2051230 msgr2=0x5556a204d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb94ffff640 1 --2- 172.21.15.102:0/637190562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2051230 0x5556a204d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb94ffff640 1 -- 172.21.15.102:0/637190562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a2053990 con 0x5556a2046b60 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb954a07640 1 --2- 172.21.15.102:0/637190562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046240 0x5556a205cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb94ffff640 1 --2- 172.21.15.102:0/637190562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a20489a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb93c000c00 tx=0x7fb93c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527bd0d8b3aa8349 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb93a7fc640 1 -- 172.21.15.102:0/637190562 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c020040 con 0x5556a2046b60 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb93a7fc640 1 -- 172.21.15.102:0/637190562 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb93c09b6f0 con 0x5556a2046b60 2024-04-24T23:14:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.468+0000 7fb93a7fc640 1 -- 172.21.15.102:0/637190562 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0ab560 con 0x5556a2046b60 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 -- 172.21.15.102:0/637190562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 msgr2=0x5556a20489a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 --2- 172.21.15.102:0/637190562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a20489a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb93c000c00 tx=0x7fb93c009ea0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 -- 172.21.15.102:0/637190562 shutdown_connections 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 --2- 172.21.15.102:0/637190562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2051230 0x5556a204d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 --2- 172.21.15.102:0/637190562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a20489a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 --2- 172.21.15.102:0/637190562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2046240 0x5556a205cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 -- 172.21.15.102:0/637190562 >> 172.21.15.102:0/637190562 conn(0x5556a2042560 msgr2=0x5556a2040d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 -- 172.21.15.102:0/637190562 shutdown_connections 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 -- 172.21.15.102:0/637190562 wait complete. 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 -- start start 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 Processor -- start 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a2059690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a2054a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2051230 0x5556a2054f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ab9c0 con 0x5556a2046b60 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ace30 con 0x5556a2046240 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ac090 con 0x5556a2051230 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.469+0000 7fb95997b740 10 client.0 _ll_register_callbacks cb 0x7fb948009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a2054a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a2054a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60652/0 (socket says 172.21.15.102:60652) 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb94ffff640 1 -- 172.21.15.102:0/1909285406 learned_addr learned my addr 172.21.15.102:0/1909285406 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb954a07640 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a2059690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb955208640 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2051230 0x5556a2054f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb94ffff640 1 -- 172.21.15.102:0/1909285406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2051230 msgr2=0x5556a2054f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb94ffff640 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2051230 0x5556a2054f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb94ffff640 1 -- 172.21.15.102:0/1909285406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 msgr2=0x5556a2059690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb94ffff640 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a2059690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb94ffff640 1 -- 172.21.15.102:0/1909285406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a20555e0 con 0x5556a2046b60 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb955208640 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2051230 0x5556a2054f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb954a07640 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a2059690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb94ffff640 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a2054a00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0980c0 tx=0x7fb93c09b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9140946208d4173 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb92ffff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb92ffff640 1 -- 172.21.15.102:0/1909285406 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c020070 con 0x5556a2046b60 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb92ffff640 1 -- 172.21.15.102:0/1909285406 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb93c0cf040 con 0x5556a2046b60 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb95997b740 1 -- 172.21.15.102:0/1909285406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb93c09b6f0 con 0x5556a2046b60 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb92ffff640 1 -- 172.21.15.102:0/1909285406 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb93c0a4360 con 0x5556a2046b60 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb95997b740 10 client.15507 fetch_fsmap learned FSMap version 23 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb95997b740 1 -- 172.21.15.102:0/1909285406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5556a20c7bd0 con 0x5556a2046b60 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb92ffff640 1 -- 172.21.15.102:0/1909285406 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb93c09b6f0 con 0x5556a2046b60 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb92ffff640 1 -- 172.21.15.102:0/1909285406 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7fb93c010660 con 0x5556a2046b60 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb95997b740 10 client.15507 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:14:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.470+0000 7fb95997b740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.471+0000 7fb95997b740 20 client.15507 populate_metadata read hostname 'smithi102' 2024-04-24T23:14:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.471+0000 7fb95997b740 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5556a20c8300 0x5556a20ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.471+0000 7fb95997b740 4 client.15507 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:14:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.471+0000 7fb95997b740 1 -- 172.21.15.102:0/1909285406 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556a1fb2540 con 0x5556a20c8300 2024-04-24T23:14:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.471+0000 7fb954a07640 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5556a20c8300 0x5556a20ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.471+0000 7fb954a07640 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5556a20c8300 0x5556a20ca720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432b60e19d1eb12e server_cookie=3fea2bff4e39140 in_seq=0 out_seq=0 2024-04-24T23:14:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.471+0000 7fb92ffff640 10 client.15507 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb92ffff640 1 -- 172.21.15.102:0/1909285406 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556a1fb2540 con 0x5556a20c8300 2024-04-24T23:14:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb92ffff640 10 client.15507 handle_command_reply: tid=0 2024-04-24T23:14:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb95997b740 1 client.15507 shutdown 2024-04-24T23:14:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb95997b740 1 -- 172.21.15.102:0/1909285406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 msgr2=0x5556a2054a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb95997b740 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a2054a00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0980c0 tx=0x7fb93c09b2a0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb95997b740 1 -- 172.21.15.102:0/1909285406 shutdown_connections 2024-04-24T23:14:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb95997b740 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556a2051230 0x5556a2054f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb95997b740 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5556a20c8300 0x5556a20ca720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb95997b740 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556a2046b60 0x5556a2054a00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb95997b740 1 --2- 172.21.15.102:0/1909285406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556a2046240 0x5556a2059690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb92ffff640 0 client.15507 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.526+0000 7fb95997b740 1 -- 172.21.15.102:0/1909285406 >> 172.21.15.102:0/1909285406 conn(0x5556a2042560 msgr2=0x5556a2040d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb95997b740 1 -- 172.21.15.102:0/1909285406 shutdown_connections 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb95997b740 1 -- 172.21.15.102:0/1909285406 wait complete. 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb95997b740 20 client.15507 trim_cache size 0 max 16384 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:14:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:14:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:14:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:14:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 -- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9400a2f50 msgr2=0x7fb9400a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9400a2f50 0x7fb9400a5410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb948009ca0 tx=0x7fb948031630 comp rx=0 tx=0).stop 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 -- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 msgr2=0x7fb950198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 0x7fb950198350 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00d950 tx=0x7fb93c00de10 comp rx=0 tx=0).stop 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 -- 172.21.15.102:0/920769852 shutdown_connections 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb950106630 0x7fb95019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9400a2f50 0x7fb9400a5410 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb950101030 0x7fb950198350 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 --2- 172.21.15.102:0/920769852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb950100670 0x7fb950197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 -- 172.21.15.102:0/920769852 >> 172.21.15.102:0/920769852 conn(0x7fb9500777c0 msgr2=0x7fb950069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 -- 172.21.15.102:0/920769852 shutdown_connections 2024-04-24T23:14:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:23.527+0000 7fb956c6b640 1 -- 172.21.15.102:0/920769852 wait complete. 2024-04-24T23:14:23.584 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/types.h... yes 2024-04-24T23:14:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:23 smithi161 ceph-mon[32709]: pgmap v43: 97 pgs: 97 active+clean; 1.5 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-24T23:14:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:23 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:14:23.653 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/stat.h... yes 2024-04-24T23:14:23.722 INFO:tasks.workunit.client.0.smithi102.stdout:checking for stdlib.h... yes 2024-04-24T23:14:23.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:23 smithi183 ceph-mon[33137]: pgmap v43: 97 pgs: 97 active+clean; 1.5 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-24T23:14:23.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:23 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:14:23.792 INFO:tasks.workunit.client.0.smithi102.stdout:checking for string.h... yes 2024-04-24T23:14:23.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:23 smithi102 ceph-mon[29216]: pgmap v43: 97 pgs: 97 active+clean; 1.5 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-24T23:14:23.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:23 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:14:23.862 INFO:tasks.workunit.client.0.smithi102.stdout:checking for memory.h... yes 2024-04-24T23:14:23.932 INFO:tasks.workunit.client.0.smithi102.stdout:checking for strings.h... yes 2024-04-24T23:14:24.002 INFO:tasks.workunit.client.0.smithi102.stdout:checking for inttypes.h... yes 2024-04-24T23:14:24.076 INFO:tasks.workunit.client.0.smithi102.stdout:checking for stdint.h... yes 2024-04-24T23:14:24.149 INFO:tasks.workunit.client.0.smithi102.stdout:checking for unistd.h... yes 2024-04-24T23:14:24.237 INFO:tasks.workunit.client.0.smithi102.stdout:checking pthread.h usability... yes 2024-04-24T23:14:24.267 INFO:tasks.workunit.client.0.smithi102.stdout:checking pthread.h presence... yes 2024-04-24T23:14:24.267 INFO:tasks.workunit.client.0.smithi102.stdout:checking for pthread.h... yes 2024-04-24T23:14:24.346 INFO:tasks.workunit.client.0.smithi102.stdout:checking fcntl.h usability... yes 2024-04-24T23:14:24.374 INFO:tasks.workunit.client.0.smithi102.stdout:checking fcntl.h presence... yes 2024-04-24T23:14:24.374 INFO:tasks.workunit.client.0.smithi102.stdout:checking for fcntl.h... yes 2024-04-24T23:14:24.451 INFO:tasks.workunit.client.0.smithi102.stdout:checking limits.h usability... yes 2024-04-24T23:14:24.478 INFO:tasks.workunit.client.0.smithi102.stdout:checking limits.h presence... yes 2024-04-24T23:14:24.479 INFO:tasks.workunit.client.0.smithi102.stdout:checking for limits.h... yes 2024-04-24T23:14:24.500 INFO:tasks.workunit.client.0.smithi102.stdout:checking for stdint.h... (cached) yes 2024-04-24T23:14:24.564 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/time.h usability... yes 2024-04-24T23:14:24.591 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/time.h presence... yes 2024-04-24T23:14:24.592 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/time.h... yes 2024-04-24T23:14:24.621 INFO:tasks.workunit.client.0.smithi102.stdout:checking for unistd.h... (cached) yes 2024-04-24T23:14:24.719 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/vfs.h usability... yes 2024-04-24T23:14:24.746 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/vfs.h presence... yes 2024-04-24T23:14:24.747 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/vfs.h... yes 2024-04-24T23:14:24.828 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/limits.h usability... no 2024-04-24T23:14:24.859 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/limits.h presence... no 2024-04-24T23:14:24.859 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/limits.h... no 2024-04-24T23:14:24.907 INFO:tasks.workunit.client.0.smithi102.stdout:checking for an ANSI C-conforming const... yes 2024-04-24T23:14:24.953 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-24T23:14:24.956 INFO:tasks.workunit.client.0.smithi102.stdout:checking for function prototypes... yes 2024-04-24T23:14:25.001 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether setvbuf arguments are reversed... no 2024-04-24T23:14:25.093 INFO:tasks.workunit.client.0.smithi102.stdout:checking for system... yes 2024-04-24T23:14:25.188 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gettimeofday... yes 2024-04-24T23:14:25.290 INFO:tasks.workunit.client.0.smithi102.stdout:checking for mkdir... yes 2024-04-24T23:14:25.380 INFO:tasks.workunit.client.0.smithi102.stdout:checking for strerror... yes 2024-04-24T23:14:25.475 INFO:tasks.workunit.client.0.smithi102.stdout:checking for open64... yes 2024-04-24T23:14:25.568 INFO:tasks.workunit.client.0.smithi102.stdout:checking for stat64... yes 2024-04-24T23:14:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:25 smithi161 ceph-mon[32709]: pgmap v44: 97 pgs: 97 active+clean; 6.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 434 KiB/s wr, 68 op/s 2024-04-24T23:14:25.668 INFO:tasks.workunit.client.0.smithi102.stdout:checking for fseeko64... yes 2024-04-24T23:14:25.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:25 smithi183 ceph-mon[33137]: pgmap v44: 97 pgs: 97 active+clean; 6.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 434 KiB/s wr, 68 op/s 2024-04-24T23:14:25.763 INFO:tasks.workunit.client.0.smithi102.stdout:checking for lrand48_r... yes 2024-04-24T23:14:25.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:25 smithi102 ceph-mon[29216]: pgmap v44: 97 pgs: 97 active+clean; 6.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 434 KiB/s wr, 68 op/s 2024-04-24T23:14:25.860 INFO:tasks.workunit.client.0.smithi102.stdout:checking for srand48_r... yes 2024-04-24T23:14:25.890 INFO:tasks.workunit.client.0.smithi102.stdout:configure: creating ./config.status 2024-04-24T23:14:26.029 INFO:tasks.workunit.client.0.smithi102.stdout:config.status: creating Makefile 2024-04-24T23:14:26.049 INFO:tasks.workunit.client.0.smithi102.stdout:config.status: creating config.h 2024-04-24T23:14:26.078 INFO:tasks.workunit.client.0.smithi102.stdout:config.status: executing depfiles commands 2024-04-24T23:14:26.148 INFO:tasks.workunit.client.0.smithi102.stdout:config.status: executing default commands 2024-04-24T23:14:26.217 INFO:tasks.workunit.client.0.smithi102.stderr:+ make 2024-04-24T23:14:26.244 INFO:tasks.workunit.client.0.smithi102.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-04-24T23:14:26.250 INFO:tasks.workunit.client.0.smithi102.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-04-24T23:14:26.251 INFO:tasks.workunit.client.0.smithi102.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-04-24T23:14:26.251 INFO:tasks.workunit.client.0.smithi102.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-04-24T23:14:26.251 INFO:tasks.workunit.client.0.smithi102.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-04-24T23:14:26.251 INFO:tasks.workunit.client.0.smithi102.stderr: any GNU archive site. 2024-04-24T23:14:26.256 INFO:tasks.workunit.client.0.smithi102.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-04-24T23:14:26.260 INFO:tasks.workunit.client.0.smithi102.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-04-24T23:14:26.261 INFO:tasks.workunit.client.0.smithi102.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-04-24T23:14:26.261 INFO:tasks.workunit.client.0.smithi102.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-04-24T23:14:26.262 INFO:tasks.workunit.client.0.smithi102.stderr: You might want to install the `Automake' and `Perl' packages. 2024-04-24T23:14:26.262 INFO:tasks.workunit.client.0.smithi102.stderr: Grab them from any GNU archive site. 2024-04-24T23:14:26.495 INFO:tasks.workunit.client.0.smithi102.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-04-24T23:14:26.839 INFO:tasks.workunit.client.0.smithi102.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-04-24T23:14:26.840 INFO:tasks.workunit.client.0.smithi102.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-04-24T23:14:26.840 INFO:tasks.workunit.client.0.smithi102.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-04-24T23:14:26.840 INFO:tasks.workunit.client.0.smithi102.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-04-24T23:14:26.972 INFO:tasks.workunit.client.0.smithi102.stdout:/bin/sh ./config.status --recheck 2024-04-24T23:14:27.040 INFO:tasks.workunit.client.0.smithi102.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-04-24T23:14:27.199 INFO:tasks.workunit.client.0.smithi102.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-24T23:14:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:27 smithi161 ceph-mon[32709]: pgmap v45: 97 pgs: 97 active+clean; 6.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 433 KiB/s wr, 64 op/s 2024-04-24T23:14:27.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:27 smithi183 ceph-mon[33137]: pgmap v45: 97 pgs: 97 active+clean; 6.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 433 KiB/s wr, 64 op/s 2024-04-24T23:14:27.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:27 smithi102 ceph-mon[29216]: pgmap v45: 97 pgs: 97 active+clean; 6.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 433 KiB/s wr, 64 op/s 2024-04-24T23:14:28.211 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether build environment is sane... yes 2024-04-24T23:14:28.223 INFO:tasks.workunit.client.0.smithi102.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-24T23:14:28.224 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gawk... gawk 2024-04-24T23:14:28.253 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether make sets $(MAKE)... yes 2024-04-24T23:14:28.303 INFO:tasks.workunit.client.0.smithi102.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-24T23:14:28.303 INFO:tasks.workunit.client.0.smithi102.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-24T23:14:28.329 INFO:tasks.workunit.client.0.smithi102.stdout:checking for style of include used by make... GNU 2024-04-24T23:14:28.336 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gcc... gcc 2024-04-24T23:14:28.464 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether the C compiler works... yes 2024-04-24T23:14:28.464 INFO:tasks.workunit.client.0.smithi102.stdout:checking for C compiler default output file name... a.out 2024-04-24T23:14:28.524 INFO:tasks.workunit.client.0.smithi102.stdout:checking for suffix of executables... 2024-04-24T23:14:28.580 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether we are cross compiling... no 2024-04-24T23:14:28.635 INFO:tasks.workunit.client.0.smithi102.stdout:checking for suffix of object files... o 2024-04-24T23:14:28.671 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether we are using the GNU C compiler... yes 2024-04-24T23:14:28.704 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether gcc accepts -g... yes 2024-04-24T23:14:28.749 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-24T23:14:28.904 INFO:tasks.workunit.client.0.smithi102.stdout:checking dependency style of gcc... gcc3 2024-04-24T23:14:28.970 INFO:tasks.workunit.client.0.smithi102.stdout:checking how to run the C preprocessor... gcc -E 2024-04-24T23:14:29.035 INFO:tasks.workunit.client.0.smithi102.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-24T23:14:29.036 INFO:tasks.workunit.client.0.smithi102.stdout:checking for egrep... /usr/bin/grep -E 2024-04-24T23:14:29.217 INFO:tasks.workunit.client.0.smithi102.stdout:checking for ANSI C header files... yes 2024-04-24T23:14:29.279 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/types.h... yes 2024-04-24T23:14:29.339 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/stat.h... yes 2024-04-24T23:14:29.402 INFO:tasks.workunit.client.0.smithi102.stdout:checking for stdlib.h... yes 2024-04-24T23:14:29.464 INFO:tasks.workunit.client.0.smithi102.stdout:checking for string.h... yes 2024-04-24T23:14:29.534 INFO:tasks.workunit.client.0.smithi102.stdout:checking for memory.h... yes 2024-04-24T23:14:29.598 INFO:tasks.workunit.client.0.smithi102.stdout:checking for strings.h... yes 2024-04-24T23:14:29.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:29 smithi161 ceph-mon[32709]: pgmap v46: 97 pgs: 97 active+clean; 8.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 705 KiB/s wr, 121 op/s 2024-04-24T23:14:29.667 INFO:tasks.workunit.client.0.smithi102.stdout:checking for inttypes.h... yes 2024-04-24T23:14:29.729 INFO:tasks.workunit.client.0.smithi102.stdout:checking for stdint.h... yes 2024-04-24T23:14:29.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:29 smithi183 ceph-mon[33137]: pgmap v46: 97 pgs: 97 active+clean; 8.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 705 KiB/s wr, 121 op/s 2024-04-24T23:14:29.789 INFO:tasks.workunit.client.0.smithi102.stdout:checking for unistd.h... yes 2024-04-24T23:14:29.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:29 smithi102 ceph-mon[29216]: pgmap v46: 97 pgs: 97 active+clean; 8.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 705 KiB/s wr, 121 op/s 2024-04-24T23:14:29.859 INFO:tasks.workunit.client.0.smithi102.stdout:checking minix/config.h usability... no 2024-04-24T23:14:29.899 INFO:tasks.workunit.client.0.smithi102.stdout:checking minix/config.h presence... no 2024-04-24T23:14:29.899 INFO:tasks.workunit.client.0.smithi102.stdout:checking for minix/config.h... no 2024-04-24T23:14:29.969 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-04-24T23:14:29.980 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gcc... (cached) gcc 2024-04-24T23:14:30.034 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-24T23:14:30.034 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-24T23:14:30.035 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-24T23:14:30.035 INFO:tasks.workunit.client.0.smithi102.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-24T23:14:30.104 INFO:tasks.workunit.client.0.smithi102.stdout:checking for main in -lm... yes 2024-04-24T23:14:30.168 INFO:tasks.workunit.client.0.smithi102.stdout:checking for main in -lpthread... yes 2024-04-24T23:14:30.174 INFO:tasks.workunit.client.0.smithi102.stdout:checking for ANSI C header files... (cached) yes 2024-04-24T23:14:30.217 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-24T23:14:30.279 INFO:tasks.workunit.client.0.smithi102.stdout:checking pthread.h usability... yes 2024-04-24T23:14:30.305 INFO:tasks.workunit.client.0.smithi102.stdout:checking pthread.h presence... yes 2024-04-24T23:14:30.305 INFO:tasks.workunit.client.0.smithi102.stdout:checking for pthread.h... yes 2024-04-24T23:14:30.376 INFO:tasks.workunit.client.0.smithi102.stdout:checking fcntl.h usability... yes 2024-04-24T23:14:30.399 INFO:tasks.workunit.client.0.smithi102.stdout:checking fcntl.h presence... yes 2024-04-24T23:14:30.400 INFO:tasks.workunit.client.0.smithi102.stdout:checking for fcntl.h... yes 2024-04-24T23:14:30.473 INFO:tasks.workunit.client.0.smithi102.stdout:checking limits.h usability... yes 2024-04-24T23:14:30.496 INFO:tasks.workunit.client.0.smithi102.stdout:checking limits.h presence... yes 2024-04-24T23:14:30.496 INFO:tasks.workunit.client.0.smithi102.stdout:checking for limits.h... yes 2024-04-24T23:14:30.522 INFO:tasks.workunit.client.0.smithi102.stdout:checking for stdint.h... (cached) yes 2024-04-24T23:14:30.580 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/time.h usability... yes 2024-04-24T23:14:30.604 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/time.h presence... yes 2024-04-24T23:14:30.604 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/time.h... yes 2024-04-24T23:14:30.624 INFO:tasks.workunit.client.0.smithi102.stdout:checking for unistd.h... (cached) yes 2024-04-24T23:14:30.681 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/vfs.h usability... yes 2024-04-24T23:14:30.703 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/vfs.h presence... yes 2024-04-24T23:14:30.704 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/vfs.h... yes 2024-04-24T23:14:30.780 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/limits.h usability... no 2024-04-24T23:14:30.824 INFO:tasks.workunit.client.0.smithi102.stdout:checking sys/limits.h presence... no 2024-04-24T23:14:30.824 INFO:tasks.workunit.client.0.smithi102.stdout:checking for sys/limits.h... no 2024-04-24T23:14:30.863 INFO:tasks.workunit.client.0.smithi102.stdout:checking for an ANSI C-conforming const... yes 2024-04-24T23:14:30.900 INFO:tasks.workunit.client.0.smithi102.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-24T23:14:30.988 INFO:tasks.workunit.client.0.smithi102.stdout:checking for system... yes 2024-04-24T23:14:31.077 INFO:tasks.workunit.client.0.smithi102.stdout:checking for gettimeofday... yes 2024-04-24T23:14:31.166 INFO:tasks.workunit.client.0.smithi102.stdout:checking for mkdir... yes 2024-04-24T23:14:31.258 INFO:tasks.workunit.client.0.smithi102.stdout:checking for strerror... yes 2024-04-24T23:14:31.346 INFO:tasks.workunit.client.0.smithi102.stdout:checking for open64... yes 2024-04-24T23:14:31.433 INFO:tasks.workunit.client.0.smithi102.stdout:checking for stat64... yes 2024-04-24T23:14:31.523 INFO:tasks.workunit.client.0.smithi102.stdout:checking for fseeko64... yes 2024-04-24T23:14:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:31 smithi161 ceph-mon[32709]: pgmap v47: 97 pgs: 97 active+clean; 8.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 688 KiB/s wr, 115 op/s 2024-04-24T23:14:31.639 INFO:tasks.workunit.client.0.smithi102.stdout:checking for lrand48_r... yes 2024-04-24T23:14:31.639 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:31 smithi102 ceph-mon[29216]: pgmap v47: 97 pgs: 97 active+clean; 8.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 688 KiB/s wr, 115 op/s 2024-04-24T23:14:31.697 INFO:tasks.workunit.client.0.smithi102.stdout:checking for srand48_r... yes 2024-04-24T23:14:31.728 INFO:tasks.workunit.client.0.smithi102.stdout:configure: creating ./config.status 2024-04-24T23:14:31.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:31 smithi183 ceph-mon[33137]: pgmap v47: 97 pgs: 97 active+clean; 8.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 688 KiB/s wr, 115 op/s 2024-04-24T23:14:31.833 INFO:tasks.workunit.client.0.smithi102.stdout: /bin/sh ./config.status 2024-04-24T23:14:31.902 INFO:tasks.workunit.client.0.smithi102.stdout:config.status: creating Makefile 2024-04-24T23:14:31.920 INFO:tasks.workunit.client.0.smithi102.stdout:config.status: creating config.h 2024-04-24T23:14:31.948 INFO:tasks.workunit.client.0.smithi102.stdout:config.status: executing depfiles commands 2024-04-24T23:14:31.966 INFO:tasks.workunit.client.0.smithi102.stdout:config.status: executing default commands 2024-04-24T23:14:32.020 INFO:tasks.workunit.client.0.smithi102.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-04-24T23:14:32.180 INFO:tasks.workunit.client.0.smithi102.stdout:rm -f stamp-h1 2024-04-24T23:14:32.183 INFO:tasks.workunit.client.0.smithi102.stdout:touch config.h.in 2024-04-24T23:14:32.185 INFO:tasks.workunit.client.0.smithi102.stdout:cd . && /bin/sh ./config.status config.h 2024-04-24T23:14:32.246 INFO:tasks.workunit.client.0.smithi102.stdout:config.status: creating config.h 2024-04-24T23:14:32.285 INFO:tasks.workunit.client.0.smithi102.stdout:make all-am 2024-04-24T23:14:32.309 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-24T23:14:32.310 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-04-24T23:14:32.518 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-04-24T23:14:32.526 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-04-24T23:14:32.687 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-04-24T23:14:32.695 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-04-24T23:14:32.858 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-04-24T23:14:32.864 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-04-24T23:14:33.014 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-04-24T23:14:33.020 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-04-24T23:14:33.199 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-04-24T23:14:33.205 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-04-24T23:14:33.316 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-04-24T23:14:33.322 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-04-24T23:14:33.402 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-04-24T23:14:33.408 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-04-24T23:14:33.506 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-04-24T23:14:33.511 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-04-24T23:14:33.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:33 smithi161 ceph-mon[32709]: pgmap v48: 97 pgs: 97 active+clean; 8.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 688 KiB/s wr, 115 op/s 2024-04-24T23:14:33.754 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-04-24T23:14:33.760 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-04-24T23:14:33.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:33 smithi183 ceph-mon[33137]: pgmap v48: 97 pgs: 97 active+clean; 8.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 688 KiB/s wr, 115 op/s 2024-04-24T23:14:33.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:33 smithi102 ceph-mon[29216]: pgmap v48: 97 pgs: 97 active+clean; 8.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 688 KiB/s wr, 115 op/s 2024-04-24T23:14:33.941 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-04-24T23:14:33.947 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-04-24T23:14:34.170 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-04-24T23:14:34.176 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-04-24T23:14:34.294 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-04-24T23:14:34.300 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-04-24T23:14:34.430 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-04-24T23:14:34.436 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-04-24T23:14:34.639 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-04-24T23:14:34.645 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-04-24T23:14:34.720 INFO:tasks.workunit.client.0.smithi102.stderr:In file included from parser.h:22, 2024-04-24T23:14:34.721 INFO:tasks.workunit.client.0.smithi102.stderr: from parser.c:25: 2024-04-24T23:14:34.721 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c: In function ‘init_filesys’: 2024-04-24T23:14:34.722 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-24T23:14:34.722 INFO:tasks.workunit.client.0.smithi102.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-24T23:14:34.722 INFO:tasks.workunit.client.0.smithi102.stderr: | ^ 2024-04-24T23:14:34.722 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~ 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: 36 | container_of(ptr, type, member) 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~ 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~ 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-24T23:14:34.723 INFO:tasks.workunit.client.0.smithi102.stderr: | ^ 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~ 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr: 36 | container_of(ptr, type, member) 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~ 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~ 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-24T23:14:34.724 INFO:tasks.workunit.client.0.smithi102.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr: | ^ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr: 36 | container_of(ptr, type, member) 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:34.725 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: | ^ 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~ 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: 36 | container_of(ptr, type, member) 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~ 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~ 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-24T23:14:34.726 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c: In function ‘init_tg_stats’: 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr: | ^ 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~ 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr: 36 | container_of(ptr, type, member) 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~ 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~ 2024-04-24T23:14:34.727 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: | ^ 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~ 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: 36 | container_of(ptr, type, member) 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~ 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~ 2024-04-24T23:14:34.728 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: | ^ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: 36 | container_of(ptr, type, member) 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~ 2024-04-24T23:14:34.729 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr: | ^ 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~ 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr: 36 | container_of(ptr, type, member) 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~ 2024-04-24T23:14:34.730 INFO:tasks.workunit.client.0.smithi102.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-24T23:14:34.731 INFO:tasks.workunit.client.0.smithi102.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-24T23:14:34.731 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~ 2024-04-24T23:14:34.731 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-24T23:14:34.731 INFO:tasks.workunit.client.0.smithi102.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-24T23:14:34.731 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:34.777 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c: In function ‘get_optstr’: 2024-04-24T23:14:34.777 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-04-24T23:14:34.777 INFO:tasks.workunit.client.0.smithi102.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-04-24T23:14:34.777 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:34.777 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:200:14: note: source object declared here 2024-04-24T23:14:34.777 INFO:tasks.workunit.client.0.smithi102.stderr: 200 | char temp[BUFSIZE]; 2024-04-24T23:14:34.777 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~ 2024-04-24T23:14:34.783 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c: In function ‘size64_convert’: 2024-04-24T23:14:34.783 INFO:tasks.workunit.client.0.smithi102.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-04-24T23:14:34.783 INFO:tasks.workunit.client.0.smithi102.stderr: 119 | memcpy(unit, "\0", 3); 2024-04-24T23:14:34.783 INFO:tasks.workunit.client.0.smithi102.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-24T23:14:35.081 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-04-24T23:14:35.087 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-04-24T23:14:35.197 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-04-24T23:14:35.203 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-04-24T23:14:35.343 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-04-24T23:14:35.350 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-04-24T23:14:35.407 INFO:tasks.workunit.client.0.smithi102.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-04-24T23:14:35.413 INFO:tasks.workunit.client.0.smithi102.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-04-24T23:14:35.521 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-24T23:14:35.522 INFO:tasks.workunit.client.0.smithi102.stderr:+ cd .. 2024-04-24T23:14:35.523 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir tmp 2024-04-24T23:14:35.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ cd tmp 2024-04-24T23:14:35.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in $mydir/*.ffsb 2024-04-24T23:14:35.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-04-24T23:14:35.528 INFO:tasks.workunit.client.0.smithi102.stdout:FFSB version 6.0-RC2 started 2024-04-24T23:14:35.528 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:14:35.531 INFO:tasks.workunit.client.0.smithi102.stdout:benchmark time = 300 2024-04-24T23:14:35.531 INFO:tasks.workunit.client.0.smithi102.stdout:ThreadGroup 0 2024-04-24T23:14:35.531 INFO:tasks.workunit.client.0.smithi102.stdout:================ 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: num_threads = 32 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: read_random = off 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: read_size = 0 (0B) 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: read_blocksize = 0 (0B) 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: read_skip = off 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: read_skipsize = 0 (0B) 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: write_random = on 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: write_size = 5242880 (5MB) 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: fsync_file = 0 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: write_blocksize = 4096 (4KB) 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: wait time = 0 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:14:35.532 INFO:tasks.workunit.client.0.smithi102.stdout: op weights 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: read = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: readall = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: write = 1 (100.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: create = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: append = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: delete = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: metaop = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: createdir = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: stat = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: writeall = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: writeall_fsync = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: open_close = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: write_fsync = 0 (0.00%) 2024-04-24T23:14:35.533 INFO:tasks.workunit.client.0.smithi102.stdout: create_fsync = 0 (0.00%) 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: append_fsync = 0 (0.00%) 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout:FileSystem . 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout:========== 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: num_dirs = 0 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: starting files = 128 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: min file size = 104857600 (100MB) 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: max file size = 104857600 (100MB) 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: directio = off 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: alignedio = on 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: bufferedio = off 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: aging is off 2024-04-24T23:14:35.534 INFO:tasks.workunit.client.0.smithi102.stdout: current utilization = 0.00% 2024-04-24T23:14:35.535 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:14:35.535 INFO:tasks.workunit.client.0.smithi102.stdout:checking existing fs: . 2024-04-24T23:14:35.535 INFO:tasks.workunit.client.0.smithi102.stdout:recreating new fileset 2024-04-24T23:14:35.535 INFO:tasks.workunit.client.0.smithi102.stderr:opendir: No such file or directory 2024-04-24T23:14:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:35 smithi161 ceph-mon[32709]: pgmap v49: 97 pgs: 97 active+clean; 11 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 962 KiB/s wr, 184 op/s 2024-04-24T23:14:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:35 smithi161 ceph-mon[32709]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-24T23:14:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:35 smithi183 ceph-mon[33137]: pgmap v49: 97 pgs: 97 active+clean; 11 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 962 KiB/s wr, 184 op/s 2024-04-24T23:14:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:35 smithi183 ceph-mon[33137]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-24T23:14:35.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:35 smithi102 ceph-mon[29216]: pgmap v49: 97 pgs: 97 active+clean; 11 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 962 KiB/s wr, 184 op/s 2024-04-24T23:14:35.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:35 smithi102 ceph-mon[29216]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-24T23:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 -- 172.21.15.102:0/3146807705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c100640 msgr2=0x7f112c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 --2- 172.21.15.102:0/3146807705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c100640 0x7f112c100ac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1120009930 tx=0x7f112002a420 comp rx=0 tx=0).stop 2024-04-24T23:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 -- 172.21.15.102:0/3146807705 shutdown_connections 2024-04-24T23:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 --2- 172.21.15.102:0/3146807705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c101000 0x7f112c10d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 --2- 172.21.15.102:0/3146807705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c100640 0x7f112c100ac0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 --2- 172.21.15.102:0/3146807705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c106650 0x7f112c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 -- 172.21.15.102:0/3146807705 >> 172.21.15.102:0/3146807705 conn(0x7f112c0777c0 msgr2=0x7f112c077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 -- 172.21.15.102:0/3146807705 shutdown_connections 2024-04-24T23:14:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 -- 172.21.15.102:0/3146807705 wait complete. 2024-04-24T23:14:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 Processor -- start 2024-04-24T23:14:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.626+0000 7f1131ada640 1 -- start start 2024-04-24T23:14:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f1131ada640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c100640 0x7f112c0fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f1131ada640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c101000 0x7f112c0fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f1131ada640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c106650 0x7f112c0fe020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f1131ada640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f112c110910 con 0x7f112c101000 2024-04-24T23:14:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f1131ada640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f112c110790 con 0x7f112c100640 2024-04-24T23:14:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f1131ada640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f112c110a90 con 0x7f112c106650 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c101000 0x7f112c0fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c101000 0x7f112c0fdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42014/0 (socket says 172.21.15.102:42014) 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112affd640 1 -- 172.21.15.102:0/2719326015 learned_addr learned my addr 172.21.15.102:0/2719326015 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112b7fe640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c100640 0x7f112c0fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112affd640 1 -- 172.21.15.102:0/2719326015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c106650 msgr2=0x7f112c0fe020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112affd640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c106650 0x7f112c0fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112affd640 1 -- 172.21.15.102:0/2719326015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c100640 msgr2=0x7f112c0fd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112affd640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c100640 0x7f112c0fd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112affd640 1 -- 172.21.15.102:0/2719326015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c1a6410 con 0x7f112c101000 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f112affd640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c101000 0x7f112c0fdae0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1120004770 tx=0x7f11200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.627+0000 7f1128ff9640 1 -- 172.21.15.102:0/2719326015 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112003f070 con 0x7f112c101000 2024-04-24T23:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.628+0000 7f1128ff9640 1 -- 172.21.15.102:0/2719326015 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f112003b460 con 0x7f112c101000 2024-04-24T23:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.628+0000 7f1128ff9640 1 -- 172.21.15.102:0/2719326015 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120035070 con 0x7f112c101000 2024-04-24T23:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.628+0000 7f1131ada640 1 -- 172.21.15.102:0/2719326015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112c1a66a0 con 0x7f112c101000 2024-04-24T23:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.628+0000 7f1131ada640 1 -- 172.21.15.102:0/2719326015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f112c1a6b40 con 0x7f112c101000 2024-04-24T23:14:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.630+0000 7f1128ff9640 1 -- 172.21.15.102:0/2719326015 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f112003b610 con 0x7f112c101000 2024-04-24T23:14:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.630+0000 7f11347ee740 1 Processor -- start 2024-04-24T23:14:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.630+0000 7f11347ee740 1 -- start start 2024-04-24T23:14:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.630+0000 7f11347ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba077620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.630+0000 7f11347ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 0x5584ba081cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.630+0000 7f11347ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba084620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.630+0000 7f11347ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5584b9fdbcb0 con 0x5584ba082230 2024-04-24T23:14:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.630+0000 7f11347ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5584b9f845b0 con 0x5584ba077b60 2024-04-24T23:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.630+0000 7f11347ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5584b9fd7c70 con 0x5584ba077240 2024-04-24T23:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba084620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba084620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42018/0 (socket says 172.21.15.102:42018) 2024-04-24T23:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112affd640 1 -- 172.21.15.102:0/3979264032 learned_addr learned my addr 172.21.15.102:0/3979264032 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f1128ff9640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11140a2e50 0x7f11140a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112b7fe640 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 0x5584ba081cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112bfff640 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba077620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112bfff640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11140a2e50 0x7f11140a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f1128ff9640 1 -- 172.21.15.102:0/2719326015 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f11200ef7d0 con 0x7f112c101000 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112b7fe640 1 -- 172.21.15.102:0/3979264032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 msgr2=0x5584ba077620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112b7fe640 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba077620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112b7fe640 1 -- 172.21.15.102:0/3979264032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 msgr2=0x5584ba084620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112b7fe640 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba084620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112b7fe640 1 -- 172.21.15.102:0/3979264032 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b9cf0210 con 0x5584ba077b60 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112affd640 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba084620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112bfff640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11140a2e50 0x7f11140a5310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1124005fd0 tx=0x7f1124009440 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.631+0000 7f112b7fe640 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 0x5584ba081cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f111c00bb60 tx=0x7f111c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fcefa9f7c156911 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.632+0000 7f1111ffb640 1 -- 172.21.15.102:0/3979264032 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c015070 con 0x5584ba077b60 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.632+0000 7f1111ffb640 1 -- 172.21.15.102:0/3979264032 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f111c00be50 con 0x5584ba077b60 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.632+0000 7f1111ffb640 1 -- 172.21.15.102:0/3979264032 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c011440 con 0x5584ba077b60 2024-04-24T23:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.632+0000 7f11347ee740 1 -- 172.21.15.102:0/3979264032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 msgr2=0x5584ba081cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.632+0000 7f11347ee740 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 0x5584ba081cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f111c00bb60 tx=0x7f111c00f730 comp rx=0 tx=0).stop 2024-04-24T23:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 -- 172.21.15.102:0/3979264032 shutdown_connections 2024-04-24T23:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba084620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 0x5584ba081cf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 --2- 172.21.15.102:0/3979264032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba077620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 -- 172.21.15.102:0/3979264032 >> 172.21.15.102:0/3979264032 conn(0x5584ba073560 msgr2=0x5584ba073940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 -- 172.21.15.102:0/3979264032 shutdown_connections 2024-04-24T23:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 -- 172.21.15.102:0/3979264032 wait complete. 2024-04-24T23:14:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 -- start start 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 Processor -- start 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba07f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 0x5584ba07f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.633+0000 7f11347ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f11347ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5584ba085020 con 0x5584ba082230 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f11347ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5584b9f97b50 con 0x5584ba077b60 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f11347ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5584ba0851a0 con 0x5584ba077240 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f11347ee740 10 client.0 _ll_register_callbacks cb 0x5584b9fd2ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42022/0 (socket says 172.21.15.102:42022) 2024-04-24T23:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112bfff640 1 -- 172.21.15.102:0/3732270627 learned_addr learned my addr 172.21.15.102:0/3732270627 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112b7fe640 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba07f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112affd640 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 0x5584ba07f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112bfff640 1 -- 172.21.15.102:0/3732270627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 msgr2=0x5584ba07f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112bfff640 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba07f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112bfff640 1 -- 172.21.15.102:0/3732270627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 msgr2=0x5584ba07f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112bfff640 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 0x5584ba07f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112bfff640 1 -- 172.21.15.102:0/3732270627 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b9dbc620 con 0x5584ba082230 2024-04-24T23:14:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112b7fe640 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba07f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f112bfff640 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07fbd0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1124003ae0 tx=0x7f1124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5d56010a011584 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f110affd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f110affd640 1 -- 172.21.15.102:0/3732270627 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124024070 con 0x5584ba082230 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f110affd640 1 -- 172.21.15.102:0/3732270627 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f1124004b20 con 0x5584ba082230 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.634+0000 7f11347ee740 1 -- 172.21.15.102:0/3732270627 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111c00be50 con 0x5584ba082230 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f110affd640 1 -- 172.21.15.102:0/3732270627 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f112402f050 con 0x5584ba082230 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f11347ee740 10 client.15519 fetch_fsmap learned FSMap version 23 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f11347ee740 1 -- 172.21.15.102:0/3732270627 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5584ba0911b0 con 0x5584ba082230 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f110affd640 1 -- 172.21.15.102:0/3732270627 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f111c00be50 con 0x5584ba082230 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f110affd640 1 -- 172.21.15.102:0/3732270627 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f112401e740 con 0x5584ba082230 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f11347ee740 10 client.15519 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f11347ee740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f11347ee740 20 client.15519 populate_metadata read hostname 'smithi102' 2024-04-24T23:14:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f11347ee740 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5584ba0926e0 0x5584ba094ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f11347ee740 4 client.15519 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f11347ee740 1 -- 172.21.15.102:0/3732270627 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584b9fe91f0 con 0x5584ba0926e0 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.635+0000 7f112b7fe640 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5584ba0926e0 0x5584ba094ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.636+0000 7f110affd640 10 client.15519 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.636+0000 7f112b7fe640 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5584ba0926e0 0x5584ba094ad0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3663e21ad593e7e5 server_cookie=2d180ae056880d7 in_seq=0 out_seq=0 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.636+0000 7f110affd640 1 -- 172.21.15.102:0/3732270627 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584b9fe91f0 con 0x5584ba0926e0 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.636+0000 7f110affd640 10 client.15519 handle_command_reply: tid=0 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.636+0000 7f11347ee740 1 client.15519 shutdown 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 -- 172.21.15.102:0/3732270627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 msgr2=0x5584ba07fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07fbd0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1124003ae0 tx=0x7f1124004a40 comp rx=0 tx=0).stop 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 -- 172.21.15.102:0/3732270627 shutdown_connections 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5584ba0926e0 0x5584ba094ad0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07fbd0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077b60 0x5584ba07f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f110affd640 0 client.15519 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 --2- 172.21.15.102:0/3732270627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba07f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 -- 172.21.15.102:0/3732270627 >> 172.21.15.102:0/3732270627 conn(0x5584ba073560 msgr2=0x5584ba079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 -- 172.21.15.102:0/3732270627 shutdown_connections 2024-04-24T23:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 1 -- 172.21.15.102:0/3732270627 wait complete. 2024-04-24T23:14:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.637+0000 7f11347ee740 20 client.15519 trim_cache size 0 max 16384 2024-04-24T23:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.651+0000 7f11347ee740 1 Processor -- start 2024-04-24T23:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f11347ee740 1 -- start start 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f11347ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba08dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f11347ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba077b60 0x5584ba0799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f11347ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba082230 0x5584ba09a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f11347ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f112c1b1e70 con 0x5584ba077b60 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f11347ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f112c1af710 con 0x5584ba082230 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f11347ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f112c1ad580 con 0x5584ba077240 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba077b60 0x5584ba0799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba077b60 0x5584ba0799a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42034/0 (socket says 172.21.15.102:42034) 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112affd640 1 -- 172.21.15.102:0/1778476864 learned_addr learned my addr 172.21.15.102:0/1778476864 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112bfff640 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba082230 0x5584ba09a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112b7fe640 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba08dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112affd640 1 -- 172.21.15.102:0/1778476864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 msgr2=0x5584ba08dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112affd640 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba08dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112affd640 1 -- 172.21.15.102:0/1778476864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba082230 msgr2=0x5584ba09a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112affd640 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba082230 0x5584ba09a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112affd640 1 -- 172.21.15.102:0/1778476864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ba077040 con 0x5584ba077b60 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112bfff640 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba082230 0x5584ba09a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.652+0000 7f112affd640 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba077b60 0x5584ba0799a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1120005940 tx=0x7f11200bbb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50d5da10cca30a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11117fa640 1 -- 172.21.15.102:0/1778476864 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120048040 con 0x5584ba077b60 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11117fa640 1 -- 172.21.15.102:0/1778476864 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11200bc3e0 con 0x5584ba077b60 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11117fa640 1 -- 172.21.15.102:0/1778476864 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11200b8350 con 0x5584ba077b60 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11347ee740 1 -- 172.21.15.102:0/1778476864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba077b60 msgr2=0x5584ba0799a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11347ee740 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba077b60 0x5584ba0799a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1120005940 tx=0x7f11200bbb70 comp rx=0 tx=0).stop 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11347ee740 1 -- 172.21.15.102:0/1778476864 shutdown_connections 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11347ee740 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba082230 0x5584ba09a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11347ee740 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba077b60 0x5584ba0799a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11347ee740 1 --2- 172.21.15.102:0/1778476864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077240 0x5584ba08dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.653+0000 7f11347ee740 1 -- 172.21.15.102:0/1778476864 >> 172.21.15.102:0/1778476864 conn(0x5584ba073560 msgr2=0x5584ba0725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 -- 172.21.15.102:0/1778476864 shutdown_connections 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 -- 172.21.15.102:0/1778476864 wait complete. 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 -- start start 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 Processor -- start 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077240 0x5584ba08a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077b60 0x5584ba07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f112c1adbd0 con 0x5584ba082230 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f112c1af040 con 0x5584ba077240 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f112c1ae2a0 con 0x5584ba077b60 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f11347ee740 10 client.0 _ll_register_callbacks cb 0x7f1124003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f112bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f112bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42046/0 (socket says 172.21.15.102:42046) 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f112bfff640 1 -- 172.21.15.102:0/1222355675 learned_addr learned my addr 172.21.15.102:0/1222355675 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.654+0000 7f112affd640 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077b60 0x5584ba07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f112b7fe640 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077240 0x5584ba08a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f112bfff640 1 -- 172.21.15.102:0/1222355675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077b60 msgr2=0x5584ba07bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f112bfff640 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077b60 0x5584ba07bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f112bfff640 1 -- 172.21.15.102:0/1222355675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077240 msgr2=0x5584ba08a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f112bfff640 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077240 0x5584ba08a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f112bfff640 1 -- 172.21.15.102:0/1222355675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ba07c7e0 con 0x5584ba082230 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f112bfff640 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07c000 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f112400f7d0 tx=0x7f1124009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0411570c0d2b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f110affd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f110affd640 1 -- 172.21.15.102:0/1222355675 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124039020 con 0x5584ba082230 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f110affd640 1 -- 172.21.15.102:0/1222355675 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f112401e860 con 0x5584ba082230 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f11347ee740 1 -- 172.21.15.102:0/1222355675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11200bc3e0 con 0x5584ba082230 2024-04-24T23:14:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f110affd640 1 -- 172.21.15.102:0/1222355675 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1124024040 con 0x5584ba082230 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f11347ee740 10 client.15525 fetch_fsmap learned FSMap version 23 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f11347ee740 1 -- 172.21.15.102:0/1222355675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5584ba0f8b90 con 0x5584ba082230 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.655+0000 7f110affd640 1 -- 172.21.15.102:0/1222355675 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11200bc3e0 con 0x5584ba082230 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f110affd640 1 -- 172.21.15.102:0/1222355675 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f1124035720 con 0x5584ba082230 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f11347ee740 10 client.15525 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f11347ee740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f11347ee740 20 client.15525 populate_metadata read hostname 'smithi102' 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f11347ee740 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5584ba0f9230 0x5584ba0fb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f11347ee740 4 client.15525 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f11347ee740 1 -- 172.21.15.102:0/1222355675 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584b9fe3540 con 0x5584ba0f9230 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f112b7fe640 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5584ba0f9230 0x5584ba0fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f110affd640 10 client.15525 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.656+0000 7f112b7fe640 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5584ba0f9230 0x5584ba0fb650 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b26f8cccab74b1 server_cookie=54e9fdbec87d73e9 in_seq=0 out_seq=0 2024-04-24T23:14:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f110affd640 1 -- 172.21.15.102:0/1222355675 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584b9fe3540 con 0x5584ba0f9230 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f110affd640 10 client.15525 handle_command_reply: tid=0 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 client.15525 shutdown 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 -- 172.21.15.102:0/1222355675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 msgr2=0x5584ba07c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07c000 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f112400f7d0 tx=0x7f1124009f80 comp rx=0 tx=0).stop 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 -- 172.21.15.102:0/1222355675 shutdown_connections 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5584ba0f9230 0x5584ba0fb650 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5584ba082230 0x5584ba07c000 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ba077b60 0x5584ba07bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f110affd640 0 client.15525 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 --2- 172.21.15.102:0/1222355675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584ba077240 0x5584ba08a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 -- 172.21.15.102:0/1222355675 >> 172.21.15.102:0/1222355675 conn(0x5584ba073560 msgr2=0x5584ba07f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 -- 172.21.15.102:0/1222355675 shutdown_connections 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 1 -- 172.21.15.102:0/1222355675 wait complete. 2024-04-24T23:14:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.665+0000 7f11347ee740 20 client.15525 trim_cache size 0 max 16384 2024-04-24T23:14:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:14:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:14:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:14:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:14:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 -- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11140a2e50 msgr2=0x7f11140a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11140a2e50 0x7f11140a5310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1124005fd0 tx=0x7f1124009440 comp rx=0 tx=0).stop 2024-04-24T23:14:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 -- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c101000 msgr2=0x7f112c0fdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c101000 0x7f112c0fdae0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1120004770 tx=0x7f11200042f0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 -- 172.21.15.102:0/2719326015 shutdown_connections 2024-04-24T23:14:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f112c106650 0x7f112c0fe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11140a2e50 0x7f11140a5310 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f112c101000 0x7f112c0fdae0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 --2- 172.21.15.102:0/2719326015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c100640 0x7f112c0fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 -- 172.21.15.102:0/2719326015 >> 172.21.15.102:0/2719326015 conn(0x7f112c0777c0 msgr2=0x7f112c06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 -- 172.21.15.102:0/2719326015 shutdown_connections 2024-04-24T23:14:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:36.666+0000 7f1131ada640 1 -- 172.21.15.102:0/2719326015 wait complete. 2024-04-24T23:14:37.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:37 smithi183 ceph-mon[33137]: pgmap v50: 97 pgs: 97 active+clean; 11 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 545 KiB/s wr, 124 op/s 2024-04-24T23:14:37.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:37 smithi102 ceph-mon[29216]: pgmap v50: 97 pgs: 97 active+clean; 11 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 545 KiB/s wr, 124 op/s 2024-04-24T23:14:37.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:37 smithi161 ceph-mon[32709]: pgmap v50: 97 pgs: 97 active+clean; 11 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 545 KiB/s wr, 124 op/s 2024-04-24T23:14:39.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:39 smithi183 ceph-mon[33137]: pgmap v51: 97 pgs: 97 active+clean; 64 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 5.0 MiB/s wr, 148 op/s 2024-04-24T23:14:39.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:39 smithi102 ceph-mon[29216]: pgmap v51: 97 pgs: 97 active+clean; 64 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 5.0 MiB/s wr, 148 op/s 2024-04-24T23:14:39.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:39 smithi161 ceph-mon[32709]: pgmap v51: 97 pgs: 97 active+clean; 64 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 5.0 MiB/s wr, 148 op/s 2024-04-24T23:14:41.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:41 smithi183 ceph-mon[33137]: pgmap v52: 97 pgs: 97 active+clean; 64 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.7 MiB/s wr, 92 op/s 2024-04-24T23:14:41.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:41 smithi102 ceph-mon[29216]: pgmap v52: 97 pgs: 97 active+clean; 64 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.7 MiB/s wr, 92 op/s 2024-04-24T23:14:41.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:41 smithi161 ceph-mon[32709]: pgmap v52: 97 pgs: 97 active+clean; 64 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.7 MiB/s wr, 92 op/s 2024-04-24T23:14:43.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:43 smithi183 ceph-mon[33137]: pgmap v53: 97 pgs: 97 active+clean; 64 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.7 MiB/s wr, 92 op/s 2024-04-24T23:14:43.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:43 smithi102 ceph-mon[29216]: pgmap v53: 97 pgs: 97 active+clean; 64 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.7 MiB/s wr, 92 op/s 2024-04-24T23:14:43.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:43 smithi161 ceph-mon[32709]: pgmap v53: 97 pgs: 97 active+clean; 64 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.7 MiB/s wr, 92 op/s 2024-04-24T23:14:45.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:45 smithi183 ceph-mon[33137]: pgmap v54: 97 pgs: 97 active+clean; 220 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 18 MiB/s wr, 101 op/s 2024-04-24T23:14:45.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:45 smithi102 ceph-mon[29216]: pgmap v54: 97 pgs: 97 active+clean; 220 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 18 MiB/s wr, 101 op/s 2024-04-24T23:14:45.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:45 smithi161 ceph-mon[32709]: pgmap v54: 97 pgs: 97 active+clean; 220 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 18 MiB/s wr, 101 op/s 2024-04-24T23:14:47.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:47 smithi183 ceph-mon[33137]: pgmap v55: 97 pgs: 97 active+clean; 220 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-04-24T23:14:47.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:47 smithi102 ceph-mon[29216]: pgmap v55: 97 pgs: 97 active+clean; 220 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-04-24T23:14:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:47 smithi161 ceph-mon[32709]: pgmap v55: 97 pgs: 97 active+clean; 220 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-04-24T23:14:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:49 smithi183 ceph-mon[33137]: pgmap v56: 97 pgs: 97 active+clean; 372 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 30 MiB/s wr, 41 op/s 2024-04-24T23:14:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 -- 172.21.15.102:0/4083296515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140106630 msgr2=0x7f0140106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 --2- 172.21.15.102:0/4083296515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140106630 0x7f0140106a10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0134009930 tx=0x7f013402a420 comp rx=0 tx=0).stop 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 -- 172.21.15.102:0/4083296515 shutdown_connections 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 --2- 172.21.15.102:0/4083296515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0140101030 0x7f014010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 --2- 172.21.15.102:0/4083296515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0140100670 0x7f0140100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 --2- 172.21.15.102:0/4083296515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140106630 0x7f0140106a10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 -- 172.21.15.102:0/4083296515 >> 172.21.15.102:0/4083296515 conn(0x7f01400777c0 msgr2=0x7f0140077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 -- 172.21.15.102:0/4083296515 shutdown_connections 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 -- 172.21.15.102:0/4083296515 wait complete. 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 Processor -- start 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.762+0000 7f0146d5a640 1 -- start start 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f0146d5a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0140100670 0x7f0140198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f0146d5a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140101030 0x7f0140199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f0146d5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0140106630 0x7f01401a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f0146d5a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f01401108b0 con 0x7f0140101030 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f0146d5a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0140110730 con 0x7f0140100670 2024-04-24T23:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f0146d5a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0140110a30 con 0x7f0140106630 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140101030 0x7f0140199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140101030 0x7f0140199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32834/0 (socket says 172.21.15.102:32834) 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013ffff640 1 -- 172.21.15.102:0/645961785 learned_addr learned my addr 172.21.15.102:0/645961785 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f0144af6640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0140100670 0x7f0140198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f01452f7640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0140106630 0x7f01401a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013ffff640 1 -- 172.21.15.102:0/645961785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0140106630 msgr2=0x7f01401a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013ffff640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0140106630 0x7f01401a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013ffff640 1 -- 172.21.15.102:0/645961785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0140100670 msgr2=0x7f0140198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013ffff640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0140100670 0x7f0140198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013ffff640 1 -- 172.21.15.102:0/645961785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01401a42f0 con 0x7f0140101030 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f01452f7640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0140106630 0x7f01401a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013ffff640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140101030 0x7f0140199270 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f012c00d950 tx=0x7f012c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013dffb640 1 -- 172.21.15.102:0/645961785 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c017030 con 0x7f0140101030 2024-04-24T23:14:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013dffb640 1 -- 172.21.15.102:0/645961785 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f012c004020 con 0x7f0140101030 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f013dffb640 1 -- 172.21.15.102:0/645961785 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c0047a0 con 0x7f0140101030 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.763+0000 7f0146d5a640 1 -- 172.21.15.102:0/645961785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01401a44c0 con 0x7f0140101030 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.764+0000 7f0146d5a640 1 -- 172.21.15.102:0/645961785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01401a4a30 con 0x7f0140101030 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.766+0000 7f013dffb640 1 -- 172.21.15.102:0/645961785 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f012c0041d0 con 0x7f0140101030 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.766+0000 7f013dffb640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f01300a2e20 0x7f01300a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.766+0000 7f013dffb640 1 -- 172.21.15.102:0/645961785 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f012c0cee70 con 0x7f0140101030 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.766+0000 7f0144af6640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f01300a2e20 0x7f01300a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0149a63740 1 Processor -- start 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0149a63740 1 -- start start 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0144af6640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f01300a2e20 0x7f01300a52e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0134009900 tx=0x7f0134000bc0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0149a63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee7b650 0x55ebdee7ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0149a63740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee86100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0149a63740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee86640 0x55ebdee88a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0149a63740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ebdeddbcb0 con 0x55ebdee7bf70 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0149a63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ebded845b0 con 0x55ebdee7b650 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0149a63740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ebdedd7c70 con 0x55ebdee86640 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f013ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee86100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f013ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee86100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32836/0 (socket says 172.21.15.102:32836) 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f013ffff640 1 -- 172.21.15.102:0/2209402297 learned_addr learned my addr 172.21.15.102:0/2209402297 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f0144af6640 1 --2- 172.21.15.102:0/2209402297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee7b650 0x55ebdee7ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f013ffff640 1 -- 172.21.15.102:0/2209402297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee86640 msgr2=0x55ebdee88a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f013ffff640 1 --2- 172.21.15.102:0/2209402297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee86640 0x55ebdee88a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f013ffff640 1 -- 172.21.15.102:0/2209402297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee7b650 msgr2=0x55ebdee7ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f013ffff640 1 --2- 172.21.15.102:0/2209402297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee7b650 0x55ebdee7ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.767+0000 7f013ffff640 1 -- 172.21.15.102:0/2209402297 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebdeaf0210 con 0x55ebdee7bf70 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.768+0000 7f013ffff640 1 --2- 172.21.15.102:0/2209402297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee86100 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f012c0a3ac0 tx=0x7f012c09bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15931b7083c5ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.768+0000 7f012affd640 1 -- 172.21.15.102:0/2209402297 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c0a3ff0 con 0x55ebdee7bf70 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.768+0000 7f012affd640 1 -- 172.21.15.102:0/2209402297 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f012c0cc830 con 0x55ebdee7bf70 2024-04-24T23:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.768+0000 7f012affd640 1 -- 172.21.15.102:0/2209402297 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c097430 con 0x55ebdee7bf70 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.768+0000 7f0149a63740 1 -- 172.21.15.102:0/2209402297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 msgr2=0x55ebdee86100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.768+0000 7f0149a63740 1 --2- 172.21.15.102:0/2209402297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee86100 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f012c0a3ac0 tx=0x7f012c09bb40 comp rx=0 tx=0).stop 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 -- 172.21.15.102:0/2209402297 shutdown_connections 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 --2- 172.21.15.102:0/2209402297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee86640 0x55ebdee88a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 --2- 172.21.15.102:0/2209402297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee86100 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 --2- 172.21.15.102:0/2209402297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee7b650 0x55ebdee7ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 -- 172.21.15.102:0/2209402297 >> 172.21.15.102:0/2209402297 conn(0x55ebdee77970 msgr2=0x55ebdee77d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 -- 172.21.15.102:0/2209402297 shutdown_connections 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 -- 172.21.15.102:0/2209402297 wait complete. 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 -- start start 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 Processor -- start 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee83700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 0x55ebdee83c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee84180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ebdee893d0 con 0x55ebdee7b650 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ebded97b50 con 0x55ebdee86640 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ebdee89550 con 0x55ebdee7bf70 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0149a63740 10 client.0 _ll_register_callbacks cb 0x55ebdedd2ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.769+0000 7f0144af6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee83700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f0144af6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee83700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32840/0 (socket says 172.21.15.102:32840) 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f0144af6640 1 -- 172.21.15.102:0/806209075 learned_addr learned my addr 172.21.15.102:0/806209075 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f013ffff640 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 0x55ebdee83c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f01452f7640 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee84180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f0144af6640 1 -- 172.21.15.102:0/806209075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 msgr2=0x55ebdee83c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f0144af6640 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 0x55ebdee83c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f0144af6640 1 -- 172.21.15.102:0/806209075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 msgr2=0x55ebdee84180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f0144af6640 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee84180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f0144af6640 1 -- 172.21.15.102:0/806209075 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebdebbc620 con 0x55ebdee7b650 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f013ffff640 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 0x55ebdee83c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f0144af6640 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee83700 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0134009930 tx=0x7f0134009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d17d7f55c741e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f011ffff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f011ffff640 1 -- 172.21.15.102:0/806209075 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013404d070 con 0x55ebdee7b650 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f011ffff640 1 -- 172.21.15.102:0/806209075 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f0134003800 con 0x55ebdee7b650 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.770+0000 7f0149a63740 1 -- 172.21.15.102:0/806209075 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012c0cc830 con 0x55ebdee7b650 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f011ffff640 1 -- 172.21.15.102:0/806209075 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0134059050 con 0x55ebdee7b650 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f0149a63740 10 client.15537 fetch_fsmap learned FSMap version 23 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f0149a63740 1 -- 172.21.15.102:0/806209075 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ebdee95700 con 0x55ebdee7b650 2024-04-24T23:14:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f011ffff640 1 -- 172.21.15.102:0/806209075 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f012c0cc830 con 0x55ebdee7b650 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f011ffff640 1 -- 172.21.15.102:0/806209075 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f0134049b10 con 0x55ebdee7b650 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f0149a63740 10 client.15537 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f0149a63740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f0149a63740 20 client.15537 populate_metadata read hostname 'smithi102' 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f0149a63740 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ebdee96c10 0x55ebdee99020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f0149a63740 4 client.15537 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f0149a63740 1 -- 172.21.15.102:0/806209075 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebdede91f0 con 0x55ebdee96c10 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.771+0000 7f013ffff640 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ebdee96c10 0x55ebdee99020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.772+0000 7f011ffff640 10 client.15537 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.772+0000 7f013ffff640 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ebdee96c10 0x55ebdee99020 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c6d066721e62cc server_cookie=f5f61c1609872eca in_seq=0 out_seq=0 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f011ffff640 1 -- 172.21.15.102:0/806209075 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebdede91f0 con 0x55ebdee96c10 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f011ffff640 10 client.15537 handle_command_reply: tid=0 2024-04-24T23:14:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 client.15537 shutdown 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 -- 172.21.15.102:0/806209075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 msgr2=0x55ebdee83700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee83700 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0134009930 tx=0x7f0134009260 comp rx=0 tx=0).stop 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 -- 172.21.15.102:0/806209075 shutdown_connections 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee84180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 0x55ebdee83c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ebdee96c10 0x55ebdee99020 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f011ffff640 0 client.15537 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 --2- 172.21.15.102:0/806209075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee83700 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 -- 172.21.15.102:0/806209075 >> 172.21.15.102:0/806209075 conn(0x55ebdee77970 msgr2=0x55ebdee7d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 -- 172.21.15.102:0/806209075 shutdown_connections 2024-04-24T23:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 1 -- 172.21.15.102:0/806209075 wait complete. 2024-04-24T23:14:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.773+0000 7f0149a63740 20 client.15537 trim_cache size 0 max 16384 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0149a63740 1 Processor -- start 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0149a63740 1 -- start start 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0149a63740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee92290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0149a63740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 0x55ebdee725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0149a63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee7ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0149a63740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f01401adae0 con 0x55ebdee7b650 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0149a63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f01401ab380 con 0x55ebdee86640 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0149a63740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01401a91f0 con 0x55ebdee7bf70 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0144af6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee92290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0144af6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee92290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32842/0 (socket says 172.21.15.102:32842) 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0144af6640 1 -- 172.21.15.102:0/2252851061 learned_addr learned my addr 172.21.15.102:0/2252851061 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f01452f7640 1 --2- 172.21.15.102:0/2252851061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee7ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f013ffff640 1 --2- 172.21.15.102:0/2252851061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 0x55ebdee725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0144af6640 1 -- 172.21.15.102:0/2252851061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 msgr2=0x55ebdee725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0144af6640 1 --2- 172.21.15.102:0/2252851061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 0x55ebdee725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0144af6640 1 -- 172.21.15.102:0/2252851061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 msgr2=0x55ebdee7ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0144af6640 1 --2- 172.21.15.102:0/2252851061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee7ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.791+0000 7f0144af6640 1 -- 172.21.15.102:0/2252851061 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebdee826b0 con 0x55ebdee7b650 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.792+0000 7f0144af6640 1 --2- 172.21.15.102:0/2252851061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee92290 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0134009230 tx=0x7f0134005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141b2b76b718cbad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.792+0000 7f012a7fc640 1 -- 172.21.15.102:0/2252851061 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134061070 con 0x55ebdee7b650 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.792+0000 7f012a7fc640 1 -- 172.21.15.102:0/2252851061 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0134049450 con 0x55ebdee7b650 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.792+0000 7f012a7fc640 1 -- 172.21.15.102:0/2252851061 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013404d040 con 0x55ebdee7b650 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.792+0000 7f0149a63740 1 -- 172.21.15.102:0/2252851061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 msgr2=0x55ebdee92290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.792+0000 7f0149a63740 1 --2- 172.21.15.102:0/2252851061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee92290 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0134009230 tx=0x7f0134005fd0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.792+0000 7f0149a63740 1 -- 172.21.15.102:0/2252851061 shutdown_connections 2024-04-24T23:14:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 --2- 172.21.15.102:0/2252851061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee7ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 --2- 172.21.15.102:0/2252851061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7bf70 0x55ebdee725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 --2- 172.21.15.102:0/2252851061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7b650 0x55ebdee92290 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 -- 172.21.15.102:0/2252851061 >> 172.21.15.102:0/2252851061 conn(0x55ebdee77970 msgr2=0x55ebdee77d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 -- 172.21.15.102:0/2252851061 shutdown_connections 2024-04-24T23:14:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 -- 172.21.15.102:0/2252851061 wait complete. 2024-04-24T23:14:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 -- start start 2024-04-24T23:14:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 Processor -- start 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7b650 0x55ebdee80210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee80750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee80c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f01401a9840 con 0x55ebdee7bf70 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f01401aacb0 con 0x55ebdee86640 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01401a9f10 con 0x55ebdee7b650 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f0149a63740 10 client.0 _ll_register_callbacks cb 0x7f0134040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.793+0000 7f013ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee80750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f013ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee80750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:32854/0 (socket says 172.21.15.102:32854) 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f013ffff640 1 -- 172.21.15.102:0/3460246853 learned_addr learned my addr 172.21.15.102:0/3460246853 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f0144af6640 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7b650 0x55ebdee80210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f01452f7640 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee80c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f013ffff640 1 -- 172.21.15.102:0/3460246853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7b650 msgr2=0x55ebdee80210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f013ffff640 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7b650 0x55ebdee80210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f013ffff640 1 -- 172.21.15.102:0/3460246853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 msgr2=0x55ebdee80c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f013ffff640 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee80c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f013ffff640 1 -- 172.21.15.102:0/3460246853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebdee88da0 con 0x55ebdee7bf70 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f01452f7640 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee80c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f0144af6640 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7b650 0x55ebdee80210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f013ffff640 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee80750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f012c09ae50 tx=0x7f012c0970d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc33a99616cd33c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f011ffff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f011ffff640 1 -- 172.21.15.102:0/3460246853 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c084070 con 0x55ebdee7bf70 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f011ffff640 1 -- 172.21.15.102:0/3460246853 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f012c0979f0 con 0x55ebdee7bf70 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f0149a63740 1 -- 172.21.15.102:0/3460246853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0134049450 con 0x55ebdee7bf70 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f011ffff640 1 -- 172.21.15.102:0/3460246853 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f012c0b4200 con 0x55ebdee7bf70 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f0149a63740 10 client.15543 fetch_fsmap learned FSMap version 23 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f0149a63740 1 -- 172.21.15.102:0/3460246853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ebdeefd1a0 con 0x55ebdee7bf70 2024-04-24T23:14:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.794+0000 7f011ffff640 1 -- 172.21.15.102:0/3460246853 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0134049450 con 0x55ebdee7bf70 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f011ffff640 1 -- 172.21.15.102:0/3460246853 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f012c009a30 con 0x55ebdee7bf70 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f0149a63740 10 client.15543 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f0149a63740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f0149a63740 20 client.15543 populate_metadata read hostname 'smithi102' 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f0149a63740 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ebdee96c10 0x55ebdeeff560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f0149a63740 4 client.15543 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f0149a63740 1 -- 172.21.15.102:0/3460246853 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ebdede3540 con 0x55ebdee96c10 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f0144af6640 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ebdee96c10 0x55ebdeeff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f011ffff640 10 client.15543 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.795+0000 7f0144af6640 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ebdee96c10 0x55ebdeeff560 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc080f6b7a3c4c3b server_cookie=ba02545ff5dc4fbd in_seq=0 out_seq=0 2024-04-24T23:14:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.803+0000 7f011ffff640 1 -- 172.21.15.102:0/3460246853 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ebdede3540 con 0x55ebdee96c10 2024-04-24T23:14:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.803+0000 7f011ffff640 10 client.15543 handle_command_reply: tid=0 2024-04-24T23:14:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.803+0000 7f0149a63740 1 client.15543 shutdown 2024-04-24T23:14:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 -- 172.21.15.102:0/3460246853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 msgr2=0x55ebdee80750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee80750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f012c09ae50 tx=0x7f012c0970d0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 -- 172.21.15.102:0/3460246853 shutdown_connections 2024-04-24T23:14:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ebdee86640 0x55ebdee80c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ebdee96c10 0x55ebdeeff560 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ebdee7bf70 0x55ebdee80750 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 --2- 172.21.15.102:0/3460246853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebdee7b650 0x55ebdee80210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 -- 172.21.15.102:0/3460246853 >> 172.21.15.102:0/3460246853 conn(0x55ebdee77970 msgr2=0x55ebdee83f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f011ffff640 0 client.15543 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 -- 172.21.15.102:0/3460246853 shutdown_connections 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 1 -- 172.21.15.102:0/3460246853 wait complete. 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0149a63740 20 client.15543 trim_cache size 0 max 16384 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:14:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:14:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0146d5a640 1 -- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f01300a2e20 msgr2=0x7f01300a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0146d5a640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f01300a2e20 0x7f01300a52e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0134009900 tx=0x7f0134000bc0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0146d5a640 1 -- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140101030 msgr2=0x7f0140199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.804+0000 7f0146d5a640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140101030 0x7f0140199270 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f012c00d950 tx=0x7f012c00de10 comp rx=0 tx=0).stop 2024-04-24T23:14:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.805+0000 7f0146d5a640 1 -- 172.21.15.102:0/645961785 shutdown_connections 2024-04-24T23:14:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.805+0000 7f0146d5a640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0140106630 0x7f01401a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.805+0000 7f0146d5a640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f01300a2e20 0x7f01300a52e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.805+0000 7f0146d5a640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0140101030 0x7f0140199270 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.805+0000 7f0146d5a640 1 --2- 172.21.15.102:0/645961785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0140100670 0x7f0140198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.805+0000 7f0146d5a640 1 -- 172.21.15.102:0/645961785 >> 172.21.15.102:0/645961785 conn(0x7f01400777c0 msgr2=0x7f0140103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.805+0000 7f0146d5a640 1 -- 172.21.15.102:0/645961785 shutdown_connections 2024-04-24T23:14:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:14:49.805+0000 7f0146d5a640 1 -- 172.21.15.102:0/645961785 wait complete. 2024-04-24T23:14:49.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:49 smithi102 ceph-mon[29216]: pgmap v56: 97 pgs: 97 active+clean; 372 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 30 MiB/s wr, 41 op/s 2024-04-24T23:14:49.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:49 smithi161 ceph-mon[32709]: pgmap v56: 97 pgs: 97 active+clean; 372 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 30 MiB/s wr, 41 op/s 2024-04-24T23:14:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:51 smithi183 ceph-mon[33137]: pgmap v57: 97 pgs: 97 active+clean; 372 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 16 op/s 2024-04-24T23:14:51.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:51 smithi102 ceph-mon[29216]: pgmap v57: 97 pgs: 97 active+clean; 372 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 16 op/s 2024-04-24T23:14:51.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:51 smithi161 ceph-mon[32709]: pgmap v57: 97 pgs: 97 active+clean; 372 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 16 op/s 2024-04-24T23:14:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:53 smithi183 ceph-mon[33137]: pgmap v58: 97 pgs: 97 active+clean; 372 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 16 op/s 2024-04-24T23:14:53.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:53 smithi102 ceph-mon[29216]: pgmap v58: 97 pgs: 97 active+clean; 372 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 16 op/s 2024-04-24T23:14:53.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:53 smithi161 ceph-mon[32709]: pgmap v58: 97 pgs: 97 active+clean; 372 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 16 op/s 2024-04-24T23:14:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:55 smithi183 ceph-mon[33137]: pgmap v59: 97 pgs: 97 active+clean; 524 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 26 op/s 2024-04-24T23:14:55.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:55 smithi102 ceph-mon[29216]: pgmap v59: 97 pgs: 97 active+clean; 524 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 26 op/s 2024-04-24T23:14:55.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:55 smithi161 ceph-mon[32709]: pgmap v59: 97 pgs: 97 active+clean; 524 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 26 op/s 2024-04-24T23:14:57.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:57 smithi183 ceph-mon[33137]: pgmap v60: 97 pgs: 97 active+clean; 524 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:14:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:57 smithi102 ceph-mon[29216]: pgmap v60: 97 pgs: 97 active+clean; 524 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:14:57.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:57 smithi161 ceph-mon[32709]: pgmap v60: 97 pgs: 97 active+clean; 524 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:14:59.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:14:59 smithi161 ceph-mon[32709]: pgmap v61: 97 pgs: 97 active+clean; 677 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:14:59.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:14:59 smithi183 ceph-mon[33137]: pgmap v61: 97 pgs: 97 active+clean; 677 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:14:59.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:14:59 smithi102 ceph-mon[29216]: pgmap v61: 97 pgs: 97 active+clean; 677 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:01 smithi183 ceph-mon[33137]: pgmap v62: 97 pgs: 97 active+clean; 677 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:01.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:01 smithi102 ceph-mon[29216]: pgmap v62: 97 pgs: 97 active+clean; 677 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:01.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:01 smithi161 ceph-mon[32709]: pgmap v62: 97 pgs: 97 active+clean; 677 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.910+0000 7f90263f9640 1 -- 172.21.15.102:0/3903254174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020100670 msgr2=0x7f9020100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.910+0000 7f90263f9640 1 --2- 172.21.15.102:0/3903254174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020100670 0x7f9020100af0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9014009930 tx=0x7f901402a420 comp rx=0 tx=0).stop 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.910+0000 7f90263f9640 1 -- 172.21.15.102:0/3903254174 shutdown_connections 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.910+0000 7f90263f9640 1 --2- 172.21.15.102:0/3903254174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9020101030 0x7f902010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.910+0000 7f90263f9640 1 --2- 172.21.15.102:0/3903254174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020100670 0x7f9020100af0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.910+0000 7f90263f9640 1 --2- 172.21.15.102:0/3903254174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9020106630 0x7f9020106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.910+0000 7f90263f9640 1 -- 172.21.15.102:0/3903254174 >> 172.21.15.102:0/3903254174 conn(0x7f90200777c0 msgr2=0x7f9020077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.910+0000 7f90263f9640 1 -- 172.21.15.102:0/3903254174 shutdown_connections 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.910+0000 7f90263f9640 1 -- 172.21.15.102:0/3903254174 wait complete. 2024-04-24T23:15:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f90263f9640 1 Processor -- start 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f90263f9640 1 -- start start 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f90263f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9020100670 0x7f90200fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f90263f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020101030 0x7f90200fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f90263f9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9020106630 0x7f90200fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f90263f9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90201108f0 con 0x7f9020101030 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f90263f9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9020110770 con 0x7f9020106630 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f90263f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9020110a70 con 0x7f9020100670 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f901f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020101030 0x7f90200fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f901f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020101030 0x7f90200fdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54438/0 (socket says 172.21.15.102:54438) 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f901f7fe640 1 -- 172.21.15.102:0/902584065 learned_addr learned my addr 172.21.15.102:0/902584065 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f901ffff640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9020100670 0x7f90200fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f901f7fe640 1 -- 172.21.15.102:0/902584065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9020100670 msgr2=0x7f90200fd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f9024996640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9020106630 0x7f90200fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f901f7fe640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9020100670 0x7f90200fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f901f7fe640 1 -- 172.21.15.102:0/902584065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9020106630 msgr2=0x7f90200fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f901f7fe640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9020106630 0x7f90200fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f901f7fe640 1 -- 172.21.15.102:0/902584065 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90201a6420 con 0x7f9020101030 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.911+0000 7f9024996640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9020106630 0x7f90200fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.912+0000 7f901f7fe640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020101030 0x7f90200fdb40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9014004770 tx=0x7f90140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.912+0000 7f901d7fa640 1 -- 172.21.15.102:0/902584065 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901403f070 con 0x7f9020101030 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.912+0000 7f901d7fa640 1 -- 172.21.15.102:0/902584065 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f901403b460 con 0x7f9020101030 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.912+0000 7f90263f9640 1 -- 172.21.15.102:0/902584065 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90201a66b0 con 0x7f9020101030 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.912+0000 7f901d7fa640 1 -- 172.21.15.102:0/902584065 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014035070 con 0x7f9020101030 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.912+0000 7f90263f9640 1 -- 172.21.15.102:0/902584065 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90201a6ba0 con 0x7f9020101030 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.914+0000 7f901d7fa640 1 -- 172.21.15.102:0/902584065 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f901403b610 con 0x7f9020101030 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f90291e9740 1 Processor -- start 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f90291e9740 1 -- start start 2024-04-24T23:15:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901d7fa640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f90080a2da0 0x7f90080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f90291e9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57240 0x5559a9a57620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f90291e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57b60 0x5559a9a61cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f90291e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 0x5559a9a64620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f90291e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5559a99bbcb0 con 0x5559a9a57b60 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f90291e9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5559a99645b0 con 0x5559a9a57240 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f90291e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5559a99b7c70 con 0x5559a9a62230 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f9024996640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f90080a2da0 0x7f90080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901d7fa640 1 -- 172.21.15.102:0/902584065 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f90140ef8a0 con 0x7f9020101030 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57b60 0x5559a9a61cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57b60 0x5559a9a61cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54448/0 (socket says 172.21.15.102:54448) 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901f7fe640 1 -- 172.21.15.102:0/1073047139 learned_addr learned my addr 172.21.15.102:0/1073047139 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901ffff640 1 --2- 172.21.15.102:0/1073047139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57240 0x5559a9a57620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901f7fe640 1 -- 172.21.15.102:0/1073047139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 msgr2=0x5559a9a64620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901f7fe640 1 --2- 172.21.15.102:0/1073047139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 0x5559a9a64620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901f7fe640 1 -- 172.21.15.102:0/1073047139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57240 msgr2=0x5559a9a57620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901f7fe640 1 --2- 172.21.15.102:0/1073047139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57240 0x5559a9a57620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f901f7fe640 1 -- 172.21.15.102:0/1073047139 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a96d0210 con 0x5559a9a57b60 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.915+0000 7f9024996640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f90080a2da0 0x7f90080a5260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9018007fa0 tx=0x7f9018007900 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f901f7fe640 1 --2- 172.21.15.102:0/1073047139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57b60 0x5559a9a61cf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f90140c44f0 tx=0x7f90140044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b793717b83cfbd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f90067fc640 1 -- 172.21.15.102:0/1073047139 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014048030 con 0x5559a9a57b60 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f90067fc640 1 -- 172.21.15.102:0/1073047139 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90140c0160 con 0x5559a9a57b60 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f90067fc640 1 -- 172.21.15.102:0/1073047139 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90140b8570 con 0x5559a9a57b60 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f90291e9740 1 -- 172.21.15.102:0/1073047139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57b60 msgr2=0x5559a9a61cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f90291e9740 1 --2- 172.21.15.102:0/1073047139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57b60 0x5559a9a61cf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f90140c44f0 tx=0x7f90140044e0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f90291e9740 1 -- 172.21.15.102:0/1073047139 shutdown_connections 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f90291e9740 1 --2- 172.21.15.102:0/1073047139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 0x5559a9a64620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f90291e9740 1 --2- 172.21.15.102:0/1073047139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57b60 0x5559a9a61cf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.916+0000 7f90291e9740 1 --2- 172.21.15.102:0/1073047139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57240 0x5559a9a57620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 -- 172.21.15.102:0/1073047139 >> 172.21.15.102:0/1073047139 conn(0x5559a9a53560 msgr2=0x5559a9a53940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 -- 172.21.15.102:0/1073047139 shutdown_connections 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 -- 172.21.15.102:0/1073047139 wait complete. 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 -- start start 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 Processor -- start 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a5f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57b60 0x5559a9a5f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a62230 0x5559a9a5fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5559a9a64de0 con 0x5559a9a57240 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5559a9977b50 con 0x5559a9a62230 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5559a9a64f60 con 0x5559a9a57b60 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f90291e9740 10 client.0 _ll_register_callbacks cb 0x5559a99b2ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f901ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a5f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f901ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a5f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54454/0 (socket says 172.21.15.102:54454) 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.917+0000 7f901ffff640 1 -- 172.21.15.102:0/3566802144 learned_addr learned my addr 172.21.15.102:0/3566802144 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f901f7fe640 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57b60 0x5559a9a5f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f9024996640 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a62230 0x5559a9a5fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f901ffff640 1 -- 172.21.15.102:0/3566802144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57b60 msgr2=0x5559a9a5f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f901ffff640 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57b60 0x5559a9a5f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f901ffff640 1 -- 172.21.15.102:0/3566802144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a62230 msgr2=0x5559a9a5fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f901ffff640 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a62230 0x5559a9a5fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f901ffff640 1 -- 172.21.15.102:0/3566802144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a979c620 con 0x5559a9a57240 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f9024996640 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a62230 0x5559a9a5fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f901ffff640 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a5f180 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9010009e90 tx=0x7f90100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f77f4159a95494c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f8fff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f8fff7fe640 1 -- 172.21.15.102:0/3566802144 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010012860 con 0x5559a9a57240 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f8fff7fe640 1 -- 172.21.15.102:0/3566802144 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9010012a00 con 0x5559a9a57240 2024-04-24T23:15:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f90291e9740 1 -- 172.21.15.102:0/3566802144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90140c0160 con 0x5559a9a57240 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f8fff7fe640 1 -- 172.21.15.102:0/3566802144 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9010012c00 con 0x5559a9a57240 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f90291e9740 10 client.15555 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.918+0000 7f90291e9740 1 -- 172.21.15.102:0/3566802144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5559a9a70f70 con 0x5559a9a57240 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f8fff7fe640 1 -- 172.21.15.102:0/3566802144 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90140c0160 con 0x5559a9a57240 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f8fff7fe640 1 -- 172.21.15.102:0/3566802144 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f901000f040 con 0x5559a9a57240 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f90291e9740 10 client.15555 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f90291e9740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f90291e9740 20 client.15555 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f90291e9740 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5559a9a724f0 0x5559a9a748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f90291e9740 4 client.15555 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f90291e9740 1 -- 172.21.15.102:0/3566802144 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559a99c91f0 con 0x5559a9a724f0 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f901f7fe640 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5559a9a724f0 0x5559a9a748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f8fff7fe640 10 client.15555 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.919+0000 7f901f7fe640 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5559a9a724f0 0x5559a9a748e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8abb7c4d0acfaf1b server_cookie=1cb562921ddfd4f3 in_seq=0 out_seq=0 2024-04-24T23:15:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f8fff7fe640 1 -- 172.21.15.102:0/3566802144 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559a99c91f0 con 0x5559a9a724f0 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f8fff7fe640 10 client.15555 handle_command_reply: tid=0 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f90291e9740 1 client.15555 shutdown 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f90291e9740 1 -- 172.21.15.102:0/3566802144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 msgr2=0x5559a9a5f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f90291e9740 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a5f180 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9010009e90 tx=0x7f90100117b0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f90291e9740 1 -- 172.21.15.102:0/3566802144 shutdown_connections 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f90291e9740 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a62230 0x5559a9a5fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f90291e9740 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57b60 0x5559a9a5f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f90291e9740 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5559a9a724f0 0x5559a9a748e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f90291e9740 1 --2- 172.21.15.102:0/3566802144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a5f180 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.920+0000 7f8fff7fe640 0 client.15555 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.921+0000 7f90291e9740 1 -- 172.21.15.102:0/3566802144 >> 172.21.15.102:0/3566802144 conn(0x5559a9a53560 msgr2=0x5559a9a59360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.921+0000 7f90291e9740 1 -- 172.21.15.102:0/3566802144 shutdown_connections 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.921+0000 7f90291e9740 1 -- 172.21.15.102:0/3566802144 wait complete. 2024-04-24T23:15:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.921+0000 7f90291e9740 20 client.15555 trim_cache size 0 max 16384 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.938+0000 7f90291e9740 1 Processor -- start 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.938+0000 7f90291e9740 1 -- start start 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.938+0000 7f90291e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57240 0x5559a9a6dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.938+0000 7f90291e9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 0x5559a9a59ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.938+0000 7f90291e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a62230 0x5559a9a5e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.938+0000 7f90291e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90201b1f90 con 0x5559a9a62230 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.938+0000 7f90291e9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90201af830 con 0x5559a9a57b60 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.938+0000 7f90291e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90201ad6a0 con 0x5559a9a57240 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.938+0000 7f9024996640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a62230 0x5559a9a5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9024996640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a62230 0x5559a9a5e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54464/0 (socket says 172.21.15.102:54464) 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9024996640 1 -- 172.21.15.102:0/286410208 learned_addr learned my addr 172.21.15.102:0/286410208 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f901f7fe640 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 0x5559a9a59ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9024996640 1 -- 172.21.15.102:0/286410208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57240 msgr2=0x5559a9a6dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f901ffff640 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57240 0x5559a9a6dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9024996640 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57240 0x5559a9a6dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9024996640 1 -- 172.21.15.102:0/286410208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 msgr2=0x5559a9a59ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9024996640 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 0x5559a9a59ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9024996640 1 -- 172.21.15.102:0/286410208 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a9a64990 con 0x5559a9a62230 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f901f7fe640 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 0x5559a9a59ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9024996640 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a62230 0x5559a9a5e300 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90180117b0 tx=0x7f9018011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d399ba40d4da1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9005ffb640 1 -- 172.21.15.102:0/286410208 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018019040 con 0x5559a9a62230 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9005ffb640 1 -- 172.21.15.102:0/286410208 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9018023070 con 0x5559a9a62230 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.939+0000 7f9005ffb640 1 -- 172.21.15.102:0/286410208 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901801ebb0 con 0x5559a9a62230 2024-04-24T23:15:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 -- 172.21.15.102:0/286410208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a62230 msgr2=0x5559a9a5e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a62230 0x5559a9a5e300 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90180117b0 tx=0x7f9018011000 comp rx=0 tx=0).stop 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 -- 172.21.15.102:0/286410208 shutdown_connections 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a62230 0x5559a9a5e300 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 0x5559a9a59ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 --2- 172.21.15.102:0/286410208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a57240 0x5559a9a6dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 -- 172.21.15.102:0/286410208 >> 172.21.15.102:0/286410208 conn(0x5559a9a53560 msgr2=0x5559a9a5f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 -- 172.21.15.102:0/286410208 shutdown_connections 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 -- 172.21.15.102:0/286410208 wait complete. 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 -- start start 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.940+0000 7f90291e9740 1 Processor -- start 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f90291e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a6a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f90291e9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 0x5559a9a5b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f90291e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 0x5559a9a5bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f90291e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90201adcf0 con 0x5559a9a57240 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f90291e9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90201af160 con 0x5559a9a57b60 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f90291e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90201ae3c0 con 0x5559a9a62230 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f90291e9740 10 client.0 _ll_register_callbacks cb 0x7f9010009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a6a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a6a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54468/0 (socket says 172.21.15.102:54468) 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901ffff640 1 -- 172.21.15.102:0/1610693342 learned_addr learned my addr 172.21.15.102:0/1610693342 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901f7fe640 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 0x5559a9a5b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f9024996640 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 0x5559a9a5bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901ffff640 1 -- 172.21.15.102:0/1610693342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 msgr2=0x5559a9a5bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901ffff640 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 0x5559a9a5bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901ffff640 1 -- 172.21.15.102:0/1610693342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 msgr2=0x5559a9a5b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901ffff640 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 0x5559a9a5b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901ffff640 1 -- 172.21.15.102:0/1610693342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a9a5c470 con 0x5559a9a57240 2024-04-24T23:15:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f9024996640 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 0x5559a9a5bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f901ffff640 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a6a2d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f90100093a0 tx=0x7f9010004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bd15186a76fde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f8fff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.941+0000 7f8fff7fe640 1 -- 172.21.15.102:0/1610693342 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010030020 con 0x5559a9a57240 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f8fff7fe640 1 -- 172.21.15.102:0/1610693342 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f90100050c0 con 0x5559a9a57240 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f90291e9740 1 -- 172.21.15.102:0/1610693342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9018023070 con 0x5559a9a57240 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f8fff7fe640 1 -- 172.21.15.102:0/1610693342 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f901003b050 con 0x5559a9a57240 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f90291e9740 10 client.15561 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f90291e9740 1 -- 172.21.15.102:0/1610693342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5559a9ad8690 con 0x5559a9a57240 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f8fff7fe640 1 -- 172.21.15.102:0/1610693342 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9018023070 con 0x5559a9a57240 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f8fff7fe640 1 -- 172.21.15.102:0/1610693342 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f9010022630 con 0x5559a9a57240 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f90291e9740 10 client.15561 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f90291e9740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f90291e9740 20 client.15561 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f90291e9740 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5559a9ad8ec0 0x5559a9adb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f90291e9740 4 client.15561 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:15:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f90291e9740 1 -- 172.21.15.102:0/1610693342 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5559a99c3540 con 0x5559a9ad8ec0 2024-04-24T23:15:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.942+0000 7f901f7fe640 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5559a9ad8ec0 0x5559a9adb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.943+0000 7f8fff7fe640 10 client.15561 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.943+0000 7f901f7fe640 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5559a9ad8ec0 0x5559a9adb2e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0991cf8c61941d9 server_cookie=3701a69abebc3587 in_seq=0 out_seq=0 2024-04-24T23:15:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f8fff7fe640 1 -- 172.21.15.102:0/1610693342 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5559a99c3540 con 0x5559a9ad8ec0 2024-04-24T23:15:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f8fff7fe640 10 client.15561 handle_command_reply: tid=0 2024-04-24T23:15:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 client.15561 shutdown 2024-04-24T23:15:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 -- 172.21.15.102:0/1610693342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 msgr2=0x5559a9a6a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a6a2d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f90100093a0 tx=0x7f9010004850 comp rx=0 tx=0).stop 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 -- 172.21.15.102:0/1610693342 shutdown_connections 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559a9a62230 0x5559a9a5bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9a57b60 0x5559a9a5b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5559a9ad8ec0 0x5559a9adb2e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 --2- 172.21.15.102:0/1610693342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559a9a57240 0x5559a9a6a2d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 -- 172.21.15.102:0/1610693342 >> 172.21.15.102:0/1610693342 conn(0x5559a9a53560 msgr2=0x5559a9a52440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f8fff7fe640 0 client.15561 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 -- 172.21.15.102:0/1610693342 shutdown_connections 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 1 -- 172.21.15.102:0/1610693342 wait complete. 2024-04-24T23:15:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.951+0000 7f90291e9740 20 client.15561 trim_cache size 0 max 16384 2024-04-24T23:15:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:15:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:15:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:15:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 -- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f90080a2da0 msgr2=0x7f90080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f90080a2da0 0x7f90080a5260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9018007fa0 tx=0x7f9018007900 comp rx=0 tx=0).stop 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 -- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020101030 msgr2=0x7f90200fdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020101030 0x7f90200fdb40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9014004770 tx=0x7f90140042f0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 -- 172.21.15.102:0/902584065 shutdown_connections 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9020106630 0x7f90200fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f90080a2da0 0x7f90080a5260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020101030 0x7f90200fdb40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 --2- 172.21.15.102:0/902584065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9020100670 0x7f90200fd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 -- 172.21.15.102:0/902584065 >> 172.21.15.102:0/902584065 conn(0x7f90200777c0 msgr2=0x7f9020069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 -- 172.21.15.102:0/902584065 shutdown_connections 2024-04-24T23:15:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:02.952+0000 7f90263f9640 1 -- 172.21.15.102:0/902584065 wait complete. 2024-04-24T23:15:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:03 smithi183 ceph-mon[33137]: pgmap v63: 97 pgs: 97 active+clean; 677 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:03.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:03 smithi102 ceph-mon[29216]: pgmap v63: 97 pgs: 97 active+clean; 677 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:03.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:03 smithi161 ceph-mon[32709]: pgmap v63: 97 pgs: 97 active+clean; 677 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:05.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:05 smithi183 ceph-mon[33137]: pgmap v64: 97 pgs: 97 active+clean; 825 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-04-24T23:15:05.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:05 smithi102 ceph-mon[29216]: pgmap v64: 97 pgs: 97 active+clean; 825 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-04-24T23:15:05.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:05 smithi161 ceph-mon[32709]: pgmap v64: 97 pgs: 97 active+clean; 825 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-04-24T23:15:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:07 smithi183 ceph-mon[33137]: pgmap v65: 97 pgs: 97 active+clean; 825 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-04-24T23:15:07.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:07 smithi102 ceph-mon[29216]: pgmap v65: 97 pgs: 97 active+clean; 825 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-04-24T23:15:07.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:07 smithi161 ceph-mon[32709]: pgmap v65: 97 pgs: 97 active+clean; 825 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-04-24T23:15:08.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:08 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:08.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:08 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:08 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:08 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:08 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:08 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:08 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:08 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:08 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:08 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:08 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:08 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:08 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:08 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:08.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:08 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:09 smithi183 ceph-mon[33137]: pgmap v66: 97 pgs: 97 active+clean; 973 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 MiB/s wr, 39 op/s 2024-04-24T23:15:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:09 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:09 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:09 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:09.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:09 smithi102 ceph-mon[29216]: pgmap v66: 97 pgs: 97 active+clean; 973 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 MiB/s wr, 39 op/s 2024-04-24T23:15:09.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:09 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:09.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:09 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:09.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:09 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:09.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:09 smithi161 ceph-mon[32709]: pgmap v66: 97 pgs: 97 active+clean; 973 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 MiB/s wr, 39 op/s 2024-04-24T23:15:09.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:09 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:09.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:09 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:09.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:09 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:15:11.765 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:11 smithi183 ceph-mon[33137]: pgmap v67: 97 pgs: 97 active+clean; 973 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:15:11.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:11 smithi102 ceph-mon[29216]: pgmap v67: 97 pgs: 97 active+clean; 973 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:15:11.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:11 smithi161 ceph-mon[32709]: pgmap v67: 97 pgs: 97 active+clean; 973 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:15:13.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:13 smithi183 ceph-mon[33137]: pgmap v68: 97 pgs: 97 active+clean; 973 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:15:13.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:13 smithi102 ceph-mon[29216]: pgmap v68: 97 pgs: 97 active+clean; 973 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:15:13.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:13 smithi161 ceph-mon[32709]: pgmap v68: 97 pgs: 97 active+clean; 973 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:15:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:15 smithi183 ceph-mon[33137]: pgmap v69: 97 pgs: 97 active+clean; 1.1 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:15:15.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:15 smithi102 ceph-mon[29216]: pgmap v69: 97 pgs: 97 active+clean; 1.1 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:15:15.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:15 smithi161 ceph-mon[32709]: pgmap v69: 97 pgs: 97 active+clean; 1.1 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:15:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.049+0000 7faa70f42640 1 -- 172.21.15.102:0/361776931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c107720 msgr2=0x7faa6c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.049+0000 7faa70f42640 1 --2- 172.21.15.102:0/361776931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c107720 0x7faa6c107b00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009930 tx=0x7faa5c02a420 comp rx=0 tx=0).stop 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.049+0000 7faa70f42640 1 -- 172.21.15.102:0/361776931 shutdown_connections 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.049+0000 7faa70f42640 1 --2- 172.21.15.102:0/361776931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa6c100600 0x7faa6c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.049+0000 7faa70f42640 1 --2- 172.21.15.102:0/361776931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa6c0ffc40 0x7faa6c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.049+0000 7faa70f42640 1 --2- 172.21.15.102:0/361776931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c107720 0x7faa6c107b00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.049+0000 7faa70f42640 1 -- 172.21.15.102:0/361776931 >> 172.21.15.102:0/361776931 conn(0x7faa6c076bd0 msgr2=0x7faa6c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.049+0000 7faa70f42640 1 -- 172.21.15.102:0/361776931 shutdown_connections 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.049+0000 7faa70f42640 1 -- 172.21.15.102:0/361776931 wait complete. 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa70f42640 1 Processor -- start 2024-04-24T23:15:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa70f42640 1 -- start start 2024-04-24T23:15:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa70f42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c0ffc40 0x7faa6c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa70f42640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa6c100600 0x7faa6c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa70f42640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa6c107720 0x7faa6c19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa70f42640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1119e0 con 0x7faa6c0ffc40 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa70f42640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c111860 con 0x7faa6c100600 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa70f42640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c111b60 con 0x7faa6c107720 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c0ffc40 0x7faa6c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c0ffc40 0x7faa6c197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55926/0 (socket says 172.21.15.102:55926) 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa6a59c640 1 -- 172.21.15.102:0/3769649809 learned_addr learned my addr 172.21.15.102:0/3769649809 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa6ad9d640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa6c107720 0x7faa6c19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa69d9b640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa6c100600 0x7faa6c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa6a59c640 1 -- 172.21.15.102:0/3769649809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa6c107720 msgr2=0x7faa6c19c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa6a59c640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa6c107720 0x7faa6c19c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa6a59c640 1 -- 172.21.15.102:0/3769649809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa6c100600 msgr2=0x7faa6c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa6a59c640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa6c100600 0x7faa6c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.050+0000 7faa6a59c640 1 -- 172.21.15.102:0/3769649809 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c19ce90 con 0x7faa6c0ffc40 2024-04-24T23:15:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.051+0000 7faa6a59c640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c0ffc40 0x7faa6c197ea0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faa5c005940 tx=0x7faa5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.051+0000 7faa5b7fe640 1 -- 172.21.15.102:0/3769649809 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c03f030 con 0x7faa6c0ffc40 2024-04-24T23:15:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.051+0000 7faa5b7fe640 1 -- 172.21.15.102:0/3769649809 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa5c003cb0 con 0x7faa6c0ffc40 2024-04-24T23:15:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.051+0000 7faa70f42640 1 -- 172.21.15.102:0/3769649809 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c19d120 con 0x7faa6c0ffc40 2024-04-24T23:15:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.051+0000 7faa5b7fe640 1 -- 172.21.15.102:0/3769649809 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c035070 con 0x7faa6c0ffc40 2024-04-24T23:15:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.051+0000 7faa70f42640 1 -- 172.21.15.102:0/3769649809 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c1a4a10 con 0x7faa6c0ffc40 2024-04-24T23:15:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.053+0000 7faa5b7fe640 1 -- 172.21.15.102:0/3769649809 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7faa5c042030 con 0x7faa6c0ffc40 2024-04-24T23:15:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa5b7fe640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faa540a2e20 0x7faa540a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa73c55740 1 Processor -- start 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa69d9b640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faa540a2e20 0x7faa540a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa5b7fe640 1 -- 172.21.15.102:0/3769649809 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7faa5c0ef910 con 0x7faa6c0ffc40 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa73c55740 1 -- start start 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa73c55740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fbc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa73c55740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fbcb60 0x55ca19fc6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa73c55740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fc7230 0x55ca19fc9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa73c55740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ca19f20cb0 con 0x55ca19fbc240 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa73c55740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ca19ec95b0 con 0x55ca19fc7230 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.054+0000 7faa73c55740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca19f1cc70 con 0x55ca19fbcb60 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa69d9b640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faa540a2e20 0x7faa540a52e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa60005e10 tx=0x7faa60005da0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fbc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fbc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55940/0 (socket says 172.21.15.102:55940) 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6a59c640 1 -- 172.21.15.102:0/2977085209 learned_addr learned my addr 172.21.15.102:0/2977085209 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa69d9b640 1 --2- 172.21.15.102:0/2977085209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fbcb60 0x55ca19fc6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6ad9d640 1 --2- 172.21.15.102:0/2977085209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fc7230 0x55ca19fc9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6a59c640 1 -- 172.21.15.102:0/2977085209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fbcb60 msgr2=0x55ca19fc6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6a59c640 1 --2- 172.21.15.102:0/2977085209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fbcb60 0x55ca19fc6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6a59c640 1 -- 172.21.15.102:0/2977085209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fc7230 msgr2=0x55ca19fc9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6a59c640 1 --2- 172.21.15.102:0/2977085209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fc7230 0x55ca19fc9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6a59c640 1 -- 172.21.15.102:0/2977085209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca19c35210 con 0x55ca19fbc240 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa6a59c640 1 --2- 172.21.15.102:0/2977085209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fbc620 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0b8280 tx=0x7faa5c0bba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ab53355d1b7540 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa58ff9640 1 -- 172.21.15.102:0/2977085209 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c049040 con 0x55ca19fbc240 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa58ff9640 1 -- 172.21.15.102:0/2977085209 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa5c0bbe20 con 0x55ca19fbc240 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.055+0000 7faa58ff9640 1 -- 172.21.15.102:0/2977085209 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c03b460 con 0x55ca19fbc240 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.056+0000 7faa73c55740 1 -- 172.21.15.102:0/2977085209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 msgr2=0x55ca19fbc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.056+0000 7faa73c55740 1 --2- 172.21.15.102:0/2977085209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fbc620 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0b8280 tx=0x7faa5c0bba20 comp rx=0 tx=0).stop 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.056+0000 7faa73c55740 1 -- 172.21.15.102:0/2977085209 shutdown_connections 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.056+0000 7faa73c55740 1 --2- 172.21.15.102:0/2977085209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fc7230 0x55ca19fc9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.056+0000 7faa73c55740 1 --2- 172.21.15.102:0/2977085209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fbcb60 0x55ca19fc6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.056+0000 7faa73c55740 1 --2- 172.21.15.102:0/2977085209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fbc620 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.056+0000 7faa73c55740 1 -- 172.21.15.102:0/2977085209 >> 172.21.15.102:0/2977085209 conn(0x55ca19fb8560 msgr2=0x55ca19fb8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.056+0000 7faa73c55740 1 -- 172.21.15.102:0/2977085209 shutdown_connections 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.056+0000 7faa73c55740 1 -- 172.21.15.102:0/2977085209 wait complete. 2024-04-24T23:15:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa73c55740 1 -- start start 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa73c55740 1 Processor -- start 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa73c55740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 0x55ca19fc4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa73c55740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fc4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa73c55740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fc4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa73c55740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ca19fc9de0 con 0x55ca19fbcb60 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa73c55740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ca19edcb50 con 0x55ca19fbc240 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa73c55740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca19fc9f60 con 0x55ca19fc7230 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa73c55740 10 client.0 _ll_register_callbacks cb 0x55ca19f17ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa69d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fc4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 0x55ca19fc4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa69d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fc4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55952/0 (socket says 172.21.15.102:55952) 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa69d9b640 1 -- 172.21.15.102:0/31956505 learned_addr learned my addr 172.21.15.102:0/31956505 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa6ad9d640 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fc4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa6a59c640 1 -- 172.21.15.102:0/31956505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 msgr2=0x55ca19fc4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa6a59c640 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fc4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa6a59c640 1 -- 172.21.15.102:0/31956505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 msgr2=0x55ca19fc4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa6a59c640 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fc4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa6a59c640 1 -- 172.21.15.102:0/31956505 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca19d01620 con 0x55ca19fbc240 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa69d9b640 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fc4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.057+0000 7faa6ad9d640 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fc4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa6a59c640 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 0x55ca19fc4150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0c0390 tx=0x7faa5c0b8730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6035309f07466b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa51ffb640 1 -- 172.21.15.102:0/31956505 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c049040 con 0x55ca19fbc240 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa51ffb640 1 -- 172.21.15.102:0/31956505 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7faa5c03b460 con 0x55ca19fbc240 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa73c55740 1 -- 172.21.15.102:0/31956505 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa5c0bbe20 con 0x55ca19fbc240 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa51ffb640 1 -- 172.21.15.102:0/31956505 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7faa5c0c01d0 con 0x55ca19fbc240 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa73c55740 10 client.25046 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa73c55740 1 -- 172.21.15.102:0/31956505 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ca19fc9de0 con 0x55ca19fbc240 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa51ffb640 1 -- 172.21.15.102:0/31956505 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa5c0bbe20 con 0x55ca19fbc240 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa51ffb640 1 -- 172.21.15.102:0/31956505 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7faa5c0c4600 con 0x55ca19fbc240 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa73c55740 10 client.25046 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.058+0000 7faa73c55740 10 client.25046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.059+0000 7faa73c55740 20 client.25046 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.059+0000 7faa73c55740 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ca19fd70b0 0x55ca19fd9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.059+0000 7faa73c55740 4 client.25046 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.059+0000 7faa73c55740 1 -- 172.21.15.102:0/31956505 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca19f2e1f0 con 0x55ca19fd70b0 2024-04-24T23:15:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.059+0000 7faa69d9b640 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ca19fd70b0 0x55ca19fd9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.059+0000 7faa69d9b640 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ca19fd70b0 0x55ca19fd9510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d117a6e38b3fcda server_cookie=4e1a5d3691532bf7 in_seq=0 out_seq=0 2024-04-24T23:15:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.059+0000 7faa51ffb640 10 client.25046 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa51ffb640 1 -- 172.21.15.102:0/31956505 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca19f2e1f0 con 0x55ca19fd70b0 2024-04-24T23:15:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa51ffb640 10 client.25046 handle_command_reply: tid=0 2024-04-24T23:15:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 client.25046 shutdown 2024-04-24T23:15:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 -- 172.21.15.102:0/31956505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 msgr2=0x55ca19fc4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 0x55ca19fc4150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0c0390 tx=0x7faa5c0b8730 comp rx=0 tx=0).stop 2024-04-24T23:15:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 -- 172.21.15.102:0/31956505 shutdown_connections 2024-04-24T23:15:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fc4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ca19fd70b0 0x55ca19fd9510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fc4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 --2- 172.21.15.102:0/31956505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 0x55ca19fc4150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa51ffb640 0 client.25046 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 -- 172.21.15.102:0/31956505 >> 172.21.15.102:0/31956505 conn(0x55ca19fb8560 msgr2=0x55ca19fbe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 -- 172.21.15.102:0/31956505 shutdown_connections 2024-04-24T23:15:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 1 -- 172.21.15.102:0/31956505 wait complete. 2024-04-24T23:15:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.060+0000 7faa73c55740 20 client.25046 trim_cache size 0 max 16384 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.078+0000 7faa73c55740 1 Processor -- start 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.078+0000 7faa73c55740 1 -- start start 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa73c55740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 0x55ca19fd2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa73c55740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fbe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa73c55740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fdee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa73c55740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1afc90 con 0x55ca19fbcb60 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa73c55740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1ad530 con 0x55ca19fbc240 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa73c55740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1ab3a0 con 0x55ca19fc7230 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa69d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fbe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa69d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fbe4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55960/0 (socket says 172.21.15.102:55960) 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa69d9b640 1 -- 172.21.15.102:0/3604073875 learned_addr learned my addr 172.21.15.102:0/3604073875 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa6a59c640 1 --2- 172.21.15.102:0/3604073875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 0x55ca19fd2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa69d9b640 1 -- 172.21.15.102:0/3604073875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 msgr2=0x55ca19fdee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa6ad9d640 1 --2- 172.21.15.102:0/3604073875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fdee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa69d9b640 1 --2- 172.21.15.102:0/3604073875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fdee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa69d9b640 1 -- 172.21.15.102:0/3604073875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 msgr2=0x55ca19fd2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa69d9b640 1 --2- 172.21.15.102:0/3604073875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 0x55ca19fd2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa69d9b640 1 -- 172.21.15.102:0/3604073875 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca19fd9b90 con 0x55ca19fbcb60 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa69d9b640 1 --2- 172.21.15.102:0/3604073875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fbe4e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55ca19fd9d90 tx=0x7faa60011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5d453dab67e178 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa53fff640 1 -- 172.21.15.102:0/3604073875 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600202d0 con 0x55ca19fbcb60 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa53fff640 1 -- 172.21.15.102:0/3604073875 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa60010d30 con 0x55ca19fbcb60 2024-04-24T23:15:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.079+0000 7faa53fff640 1 -- 172.21.15.102:0/3604073875 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6001ed70 con 0x55ca19fbcb60 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.080+0000 7faa73c55740 1 -- 172.21.15.102:0/3604073875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 msgr2=0x55ca19fbe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.080+0000 7faa73c55740 1 --2- 172.21.15.102:0/3604073875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fbe4e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55ca19fd9d90 tx=0x7faa60011000 comp rx=0 tx=0).stop 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.080+0000 7faa73c55740 1 -- 172.21.15.102:0/3604073875 shutdown_connections 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.080+0000 7faa73c55740 1 --2- 172.21.15.102:0/3604073875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fdee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.080+0000 7faa73c55740 1 --2- 172.21.15.102:0/3604073875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbcb60 0x55ca19fbe4e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.080+0000 7faa73c55740 1 --2- 172.21.15.102:0/3604073875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbc240 0x55ca19fd2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.080+0000 7faa73c55740 1 -- 172.21.15.102:0/3604073875 >> 172.21.15.102:0/3604073875 conn(0x55ca19fb8560 msgr2=0x55ca19fb75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.080+0000 7faa73c55740 1 -- 172.21.15.102:0/3604073875 shutdown_connections 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.080+0000 7faa73c55740 1 -- 172.21.15.102:0/3604073875 wait complete. 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa73c55740 1 -- start start 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa73c55740 1 Processor -- start 2024-04-24T23:15:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa73c55740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fc0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa73c55740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbcb60 0x55ca19fc13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa73c55740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fc1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa73c55740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1ab9f0 con 0x55ca19fbc240 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa73c55740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1ace60 con 0x55ca19fbcb60 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa73c55740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1ac0c0 con 0x55ca19fc7230 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa73c55740 10 client.0 _ll_register_callbacks cb 0x7faa64009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fc0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fc0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55962/0 (socket says 172.21.15.102:55962) 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa6a59c640 1 -- 172.21.15.102:0/2721422153 learned_addr learned my addr 172.21.15.102:0/2721422153 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa6ad9d640 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fc1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.081+0000 7faa69d9b640 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbcb60 0x55ca19fc13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa6a59c640 1 -- 172.21.15.102:0/2721422153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 msgr2=0x55ca19fc1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa6a59c640 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fc1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa6a59c640 1 -- 172.21.15.102:0/2721422153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbcb60 msgr2=0x55ca19fc13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa6a59c640 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbcb60 0x55ca19fc13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa6a59c640 1 -- 172.21.15.102:0/2721422153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca19fbd060 con 0x55ca19fbc240 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa6a59c640 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fc0e80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0bb270 tx=0x7faa5c03be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7624efcf0d660044 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa51ffb640 1 -- 172.21.15.102:0/2721422153 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c049040 con 0x55ca19fbc240 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa51ffb640 1 -- 172.21.15.102:0/2721422153 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7faa5c003800 con 0x55ca19fbc240 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa73c55740 1 -- 172.21.15.102:0/2721422153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa60010d30 con 0x55ca19fbc240 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa51ffb640 1 -- 172.21.15.102:0/2721422153 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7faa5c0a5120 con 0x55ca19fbc240 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa73c55740 10 client.15579 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa73c55740 1 -- 172.21.15.102:0/2721422153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ca19fbd060 con 0x55ca19fbc240 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa51ffb640 1 -- 172.21.15.102:0/2721422153 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa60010d30 con 0x55ca19fbc240 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.082+0000 7faa51ffb640 1 -- 172.21.15.102:0/2721422153 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7faa5c0c4ce0 con 0x55ca19fbc240 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.083+0000 7faa73c55740 10 client.15579 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.083+0000 7faa73c55740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.083+0000 7faa73c55740 20 client.15579 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.083+0000 7faa73c55740 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ca1a03dc40 0x55ca1a040030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.083+0000 7faa73c55740 4 client.15579 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.083+0000 7faa73c55740 1 -- 172.21.15.102:0/2721422153 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca19f28540 con 0x55ca1a03dc40 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.083+0000 7faa69d9b640 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ca1a03dc40 0x55ca1a040030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.083+0000 7faa51ffb640 10 client.15579 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.083+0000 7faa69d9b640 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ca1a03dc40 0x55ca1a040030 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40cf3ee6c9196b34 server_cookie=c57391c1b7018175 in_seq=0 out_seq=0 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa51ffb640 1 -- 172.21.15.102:0/2721422153 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca19f28540 con 0x55ca1a03dc40 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa51ffb640 10 client.15579 handle_command_reply: tid=0 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 client.15579 shutdown 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 -- 172.21.15.102:0/2721422153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 msgr2=0x55ca19fc0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fc0e80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0bb270 tx=0x7faa5c03be90 comp rx=0 tx=0).stop 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 -- 172.21.15.102:0/2721422153 shutdown_connections 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca19fc7230 0x55ca19fc1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca19fbcb60 0x55ca19fc13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ca1a03dc40 0x55ca1a040030 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa51ffb640 0 client.15579 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 --2- 172.21.15.102:0/2721422153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca19fbc240 0x55ca19fc0e80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 -- 172.21.15.102:0/2721422153 >> 172.21.15.102:0/2721422153 conn(0x55ca19fb8560 msgr2=0x55ca19fc40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 -- 172.21.15.102:0/2721422153 shutdown_connections 2024-04-24T23:15:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 1 -- 172.21.15.102:0/2721422153 wait complete. 2024-04-24T23:15:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.090+0000 7faa73c55740 20 client.15579 trim_cache size 0 max 16384 2024-04-24T23:15:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:15:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:15:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:15:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:15:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 -- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faa540a2e20 msgr2=0x7faa540a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faa540a2e20 0x7faa540a52e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa60005e10 tx=0x7faa60005da0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 -- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c0ffc40 msgr2=0x7faa6c197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c0ffc40 0x7faa6c197ea0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faa5c005940 tx=0x7faa5c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 -- 172.21.15.102:0/3769649809 shutdown_connections 2024-04-24T23:15:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa6c107720 0x7faa6c19c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa6c100600 0x7faa6c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faa540a2e20 0x7faa540a52e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 --2- 172.21.15.102:0/3769649809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa6c0ffc40 0x7faa6c197ea0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 -- 172.21.15.102:0/3769649809 >> 172.21.15.102:0/3769649809 conn(0x7faa6c076bd0 msgr2=0x7faa6c0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 -- 172.21.15.102:0/3769649809 shutdown_connections 2024-04-24T23:15:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:16.091+0000 7faa70f42640 1 -- 172.21.15.102:0/3769649809 wait complete. 2024-04-24T23:15:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:17 smithi183 ceph-mon[33137]: pgmap v70: 97 pgs: 97 active+clean; 1.1 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:17.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:17 smithi102 ceph-mon[29216]: pgmap v70: 97 pgs: 97 active+clean; 1.1 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:17 smithi161 ceph-mon[32709]: pgmap v70: 97 pgs: 97 active+clean; 1.1 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:19.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:19 smithi183 ceph-mon[33137]: pgmap v71: 97 pgs: 97 active+clean; 1.2 GiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 26 op/s 2024-04-24T23:15:19.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:19 smithi102 ceph-mon[29216]: pgmap v71: 97 pgs: 97 active+clean; 1.2 GiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 26 op/s 2024-04-24T23:15:19.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:19 smithi161 ceph-mon[32709]: pgmap v71: 97 pgs: 97 active+clean; 1.2 GiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 26 op/s 2024-04-24T23:15:21.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:21 smithi183 ceph-mon[33137]: pgmap v72: 97 pgs: 97 active+clean; 1.2 GiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:21.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:21 smithi102 ceph-mon[29216]: pgmap v72: 97 pgs: 97 active+clean; 1.2 GiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:21.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:21 smithi161 ceph-mon[32709]: pgmap v72: 97 pgs: 97 active+clean; 1.2 GiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:22.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:22 smithi183 ceph-mon[33137]: pgmap v73: 97 pgs: 97 active+clean; 1.2 GiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:22.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:22 smithi102 ceph-mon[29216]: pgmap v73: 97 pgs: 97 active+clean; 1.2 GiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:22.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:22 smithi161 ceph-mon[32709]: pgmap v73: 97 pgs: 97 active+clean; 1.2 GiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:25.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:24 smithi183 ceph-mon[33137]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:15:25.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:24 smithi102 ceph-mon[29216]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:15:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:24 smithi161 ceph-mon[32709]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:15:27.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:26 smithi183 ceph-mon[33137]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:15:27.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:26 smithi102 ceph-mon[29216]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:15:27.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:26 smithi161 ceph-mon[32709]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:15:29.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:28 smithi183 ceph-mon[33137]: pgmap v76: 97 pgs: 97 active+clean; 1.5 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:29.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:28 smithi102 ceph-mon[29216]: pgmap v76: 97 pgs: 97 active+clean; 1.5 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:29.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:28 smithi161 ceph-mon[32709]: pgmap v76: 97 pgs: 97 active+clean; 1.5 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.186+0000 7efd239ee640 1 -- 172.21.15.102:0/2067823037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 msgr2=0x7efd1c100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.186+0000 7efd239ee640 1 --2- 172.21.15.102:0/2067823037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 0x7efd1c100af0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009930 tx=0x7efd0c02a420 comp rx=0 tx=0).stop 2024-04-24T23:15:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.186+0000 7efd239ee640 1 -- 172.21.15.102:0/2067823037 shutdown_connections 2024-04-24T23:15:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.186+0000 7efd239ee640 1 --2- 172.21.15.102:0/2067823037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c101030 0x7efd1c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.186+0000 7efd239ee640 1 --2- 172.21.15.102:0/2067823037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 0x7efd1c100af0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.186+0000 7efd239ee640 1 --2- 172.21.15.102:0/2067823037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd1c106630 0x7efd1c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.186+0000 7efd239ee640 1 -- 172.21.15.102:0/2067823037 >> 172.21.15.102:0/2067823037 conn(0x7efd1c0777c0 msgr2=0x7efd1c077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.186+0000 7efd239ee640 1 -- 172.21.15.102:0/2067823037 shutdown_connections 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.186+0000 7efd239ee640 1 -- 172.21.15.102:0/2067823037 wait complete. 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd239ee640 1 Processor -- start 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd239ee640 1 -- start start 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd239ee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 0x7efd1c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd239ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd1c101030 0x7efd1c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd239ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c106630 0x7efd1c1a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd239ee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1108f0 con 0x7efd1c100670 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd239ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c110770 con 0x7efd1c101030 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd239ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c110a70 con 0x7efd1c106630 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd2178a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 0x7efd1c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd2178a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 0x7efd1c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39368/0 (socket says 172.21.15.102:39368) 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd2178a640 1 -- 172.21.15.102:0/3968294558 learned_addr learned my addr 172.21.15.102:0/3968294558 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd20f89640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd1c101030 0x7efd1c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd21f8b640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c106630 0x7efd1c1a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd2178a640 1 -- 172.21.15.102:0/3968294558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c106630 msgr2=0x7efd1c1a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd2178a640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c106630 0x7efd1c1a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd2178a640 1 -- 172.21.15.102:0/3968294558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd1c101030 msgr2=0x7efd1c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd2178a640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd1c101030 0x7efd1c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.187+0000 7efd2178a640 1 -- 172.21.15.102:0/3968294558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c1a4370 con 0x7efd1c100670 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.188+0000 7efd2178a640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 0x7efd1c198db0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efd1800d950 tx=0x7efd1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.188+0000 7efd127fc640 1 -- 172.21.15.102:0/3968294558 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18017070 con 0x7efd1c100670 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.188+0000 7efd239ee640 1 -- 172.21.15.102:0/3968294558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c1a4540 con 0x7efd1c100670 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.188+0000 7efd127fc640 1 -- 172.21.15.102:0/3968294558 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd18009c30 con 0x7efd1c100670 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.188+0000 7efd127fc640 1 -- 172.21.15.102:0/3968294558 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1800f620 con 0x7efd1c100670 2024-04-24T23:15:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.188+0000 7efd239ee640 1 -- 172.21.15.102:0/3968294558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c1a4ab0 con 0x7efd1c100670 2024-04-24T23:15:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.190+0000 7efd127fc640 1 -- 172.21.15.102:0/3968294558 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7efd180109a0 con 0x7efd1c100670 2024-04-24T23:15:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.190+0000 7efd267de740 1 Processor -- start 2024-04-24T23:15:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd267de740 1 -- start start 2024-04-24T23:15:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd267de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ea5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd267de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140eafcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd267de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140eb0230 0x559140eb2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd267de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559140e09cb0 con 0x559140ea5240 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd267de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559140db25b0 con 0x559140ea5b60 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd267de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559140e05c70 con 0x559140eb0230 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd21f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ea5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd21f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ea5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39370/0 (socket says 172.21.15.102:39370) 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd21f8b640 1 -- 172.21.15.102:0/1908989311 learned_addr learned my addr 172.21.15.102:0/1908989311 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd2178a640 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140eafcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd127fc640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efd080a2da0 0x7efd080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd20f89640 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140eb0230 0x559140eb2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd21f8b640 1 -- 172.21.15.102:0/1908989311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140eb0230 msgr2=0x559140eb2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd21f8b640 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140eb0230 0x559140eb2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd21f8b640 1 -- 172.21.15.102:0/1908989311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 msgr2=0x559140eafcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd21f8b640 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140eafcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd21f8b640 1 -- 172.21.15.102:0/1908989311 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559140b1e210 con 0x559140ea5240 2024-04-24T23:15:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd2178a640 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140eafcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd21f8b640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efd080a2da0 0x7efd080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.191+0000 7efd127fc640 1 -- 172.21.15.102:0/3968294558 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7efd180cef40 con 0x7efd1c100670 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.192+0000 7efd21f8b640 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ea5620 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd14011590 tx=0x7efd14011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d1d6df979d39b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.192+0000 7efd077fe640 1 -- 172.21.15.102:0/1908989311 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14017050 con 0x559140ea5240 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.192+0000 7efd077fe640 1 -- 172.21.15.102:0/1908989311 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd1400fcf0 con 0x559140ea5240 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.192+0000 7efd077fe640 1 -- 172.21.15.102:0/1908989311 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1401b9f0 con 0x559140ea5240 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.192+0000 7efd267de740 1 -- 172.21.15.102:0/1908989311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 msgr2=0x559140ea5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.192+0000 7efd21f8b640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efd080a2da0 0x7efd080a5260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd14022000 tx=0x7efd14023000 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.192+0000 7efd267de740 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ea5620 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd14011590 tx=0x7efd14011a50 comp rx=0 tx=0).stop 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 -- 172.21.15.102:0/1908989311 shutdown_connections 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140eb0230 0x559140eb2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140eafcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 --2- 172.21.15.102:0/1908989311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ea5620 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 -- 172.21.15.102:0/1908989311 >> 172.21.15.102:0/1908989311 conn(0x559140ea1560 msgr2=0x559140ea1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 -- 172.21.15.102:0/1908989311 shutdown_connections 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 -- 172.21.15.102:0/1908989311 wait complete. 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 -- start start 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 Processor -- start 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 0x559140ead150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5b60 0x559140ead690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140eb0230 0x559140eadbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559140eb3020 con 0x559140ea5b60 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559140dc5b50 con 0x559140eb0230 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559140eb31a0 con 0x559140ea5240 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd267de740 10 client.0 _ll_register_callbacks cb 0x559140e00ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.193+0000 7efd20f89640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5b60 0x559140ead690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd20f89640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5b60 0x559140ead690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39376/0 (socket says 172.21.15.102:39376) 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd20f89640 1 -- 172.21.15.102:0/352700538 learned_addr learned my addr 172.21.15.102:0/352700538 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd2178a640 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 0x559140ead150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd21f8b640 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140eb0230 0x559140eadbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd20f89640 1 -- 172.21.15.102:0/352700538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 msgr2=0x559140ead150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd20f89640 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 0x559140ead150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd20f89640 1 -- 172.21.15.102:0/352700538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140eb0230 msgr2=0x559140eadbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd20f89640 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140eb0230 0x559140eadbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd20f89640 1 -- 172.21.15.102:0/352700538 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559140bea620 con 0x559140ea5b60 2024-04-24T23:15:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd21f8b640 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140eb0230 0x559140eadbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd20f89640 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5b60 0x559140ead690 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efd0c02a940 tx=0x7efd0c031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177cada516e5fa70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd04ff9640 1 -- 172.21.15.102:0/352700538 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c035040 con 0x559140ea5b60 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd267de740 1 -- 172.21.15.102:0/352700538 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd1400fcf0 con 0x559140ea5b60 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd04ff9640 1 -- 172.21.15.102:0/352700538 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7efd0c0316a0 con 0x559140ea5b60 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd04ff9640 1 -- 172.21.15.102:0/352700538 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efd0c0318a0 con 0x559140ea5b60 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd267de740 10 client.15591 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.194+0000 7efd267de740 1 -- 172.21.15.102:0/352700538 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559140ebf200 con 0x559140ea5b60 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.195+0000 7efd04ff9640 1 -- 172.21.15.102:0/352700538 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd1400fcf0 con 0x559140ea5b60 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.195+0000 7efd04ff9640 1 -- 172.21.15.102:0/352700538 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7efd0c03c440 con 0x559140ea5b60 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.195+0000 7efd267de740 10 client.15591 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.195+0000 7efd267de740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.195+0000 7efd267de740 20 client.15591 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.195+0000 7efd267de740 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x559140ec0780 0x559140ec2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.195+0000 7efd267de740 4 client.15591 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.195+0000 7efd267de740 1 -- 172.21.15.102:0/352700538 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559140e171f0 con 0x559140ec0780 2024-04-24T23:15:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.195+0000 7efd2178a640 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x559140ec0780 0x559140ec2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.196+0000 7efd04ff9640 10 client.15591 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.196+0000 7efd2178a640 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x559140ec0780 0x559140ec2bd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63070c733ba1c5d server_cookie=feff726d6d68f3da in_seq=0 out_seq=0 2024-04-24T23:15:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.196+0000 7efd04ff9640 1 -- 172.21.15.102:0/352700538 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559140e171f0 con 0x559140ec0780 2024-04-24T23:15:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.196+0000 7efd04ff9640 10 client.15591 handle_command_reply: tid=0 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.196+0000 7efd267de740 1 client.15591 shutdown 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 -- 172.21.15.102:0/352700538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5b60 msgr2=0x559140ead690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5b60 0x559140ead690 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efd0c02a940 tx=0x7efd0c031e60 comp rx=0 tx=0).stop 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 -- 172.21.15.102:0/352700538 shutdown_connections 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140eb0230 0x559140eadbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x559140ec0780 0x559140ec2bd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5b60 0x559140ead690 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 --2- 172.21.15.102:0/352700538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 0x559140ead150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 -- 172.21.15.102:0/352700538 >> 172.21.15.102:0/352700538 conn(0x559140ea1560 msgr2=0x559140ea7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd04ff9640 0 client.15591 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 -- 172.21.15.102:0/352700538 shutdown_connections 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 1 -- 172.21.15.102:0/352700538 wait complete. 2024-04-24T23:15:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.197+0000 7efd267de740 20 client.15591 trim_cache size 0 max 16384 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd267de740 1 Processor -- start 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd267de740 1 -- start start 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd267de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ebbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd267de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140ea79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd267de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140eb0230 0x559140eac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd267de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1afc90 con 0x559140ea5240 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd267de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1ad530 con 0x559140ea5b60 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd267de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1ab3a0 con 0x559140eb0230 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd2178a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ebbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd2178a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ebbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39384/0 (socket says 172.21.15.102:39384) 2024-04-24T23:15:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.214+0000 7efd2178a640 1 -- 172.21.15.102:0/4028303406 learned_addr learned my addr 172.21.15.102:0/4028303406 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd20f89640 1 --2- 172.21.15.102:0/4028303406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140ea79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd2178a640 1 -- 172.21.15.102:0/4028303406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140eb0230 msgr2=0x559140eac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd2178a640 1 --2- 172.21.15.102:0/4028303406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140eb0230 0x559140eac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd2178a640 1 -- 172.21.15.102:0/4028303406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 msgr2=0x559140ea79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd2178a640 1 --2- 172.21.15.102:0/4028303406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140ea79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd2178a640 1 -- 172.21.15.102:0/4028303406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559140ea4850 con 0x559140ea5240 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd20f89640 1 --2- 172.21.15.102:0/4028303406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140ea79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd2178a640 1 --2- 172.21.15.102:0/4028303406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ebbee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efd1809b200 tx=0x7efd180d14c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79755cf22a80446 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd06ffd640 1 -- 172.21.15.102:0/4028303406 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18020040 con 0x559140ea5240 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd06ffd640 1 -- 172.21.15.102:0/4028303406 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd180a40c0 con 0x559140ea5240 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.215+0000 7efd06ffd640 1 -- 172.21.15.102:0/4028303406 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd180d10b0 con 0x559140ea5240 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 -- 172.21.15.102:0/4028303406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 msgr2=0x559140ebbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 --2- 172.21.15.102:0/4028303406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ebbee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efd1809b200 tx=0x7efd180d14c0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 -- 172.21.15.102:0/4028303406 shutdown_connections 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 --2- 172.21.15.102:0/4028303406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140eb0230 0x559140eac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 --2- 172.21.15.102:0/4028303406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140ea79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 --2- 172.21.15.102:0/4028303406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140ea5240 0x559140ebbee0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 -- 172.21.15.102:0/4028303406 >> 172.21.15.102:0/4028303406 conn(0x559140ea1560 msgr2=0x559140e9fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 -- 172.21.15.102:0/4028303406 shutdown_connections 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 -- 172.21.15.102:0/4028303406 wait complete. 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 -- start start 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.216+0000 7efd267de740 1 Processor -- start 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd267de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 0x559140eb8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd267de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140eb3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd267de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140eb0230 0x559140eb3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd267de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1ab9f0 con 0x559140eb0230 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd267de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1ace60 con 0x559140ea5b60 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd267de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1ac0c0 con 0x559140ea5240 2024-04-24T23:15:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd267de740 10 client.0 _ll_register_callbacks cb 0x7efd0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd21f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140eb0230 0x559140eb3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd21f8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140eb0230 0x559140eb3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39396/0 (socket says 172.21.15.102:39396) 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd21f8b640 1 -- 172.21.15.102:0/1552982394 learned_addr learned my addr 172.21.15.102:0/1552982394 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd20f89640 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140eb3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd2178a640 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 0x559140eb8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd21f8b640 1 -- 172.21.15.102:0/1552982394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 msgr2=0x559140eb8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd21f8b640 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 0x559140eb8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd21f8b640 1 -- 172.21.15.102:0/1552982394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 msgr2=0x559140eb3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd21f8b640 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140eb3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd21f8b640 1 -- 172.21.15.102:0/1552982394 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559140eb47b0 con 0x559140eb0230 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd21f8b640 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140eb0230 0x559140eb3fb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efd14000f80 tx=0x7efd14004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d40987c42d6e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd04ff9640 1 -- 172.21.15.102:0/1552982394 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14036060 con 0x559140eb0230 2024-04-24T23:15:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd04ff9640 1 -- 172.21.15.102:0/1552982394 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7efd1401b430 con 0x559140eb0230 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.217+0000 7efd267de740 1 -- 172.21.15.102:0/1552982394 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd180a40c0 con 0x559140eb0230 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd04ff9640 1 -- 172.21.15.102:0/1552982394 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efd14017020 con 0x559140eb0230 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd267de740 10 client.15597 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd267de740 1 -- 172.21.15.102:0/1552982394 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559140f26c20 con 0x559140eb0230 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd04ff9640 1 -- 172.21.15.102:0/1552982394 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd180a40c0 con 0x559140eb0230 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd04ff9640 1 -- 172.21.15.102:0/1552982394 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7efd1401a070 con 0x559140eb0230 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd267de740 10 client.15597 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd267de740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd267de740 20 client.15597 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd267de740 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x559140f27280 0x559140f296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd267de740 4 client.15597 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:15:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd267de740 1 -- 172.21.15.102:0/1552982394 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559140e11540 con 0x559140f27280 2024-04-24T23:15:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.218+0000 7efd2178a640 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x559140f27280 0x559140f296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.219+0000 7efd04ff9640 10 client.15597 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.219+0000 7efd2178a640 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x559140f27280 0x559140f296a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d43ff0b813d931 server_cookie=f84fc6d899d414ec in_seq=0 out_seq=0 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd04ff9640 1 -- 172.21.15.102:0/1552982394 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559140e11540 con 0x559140f27280 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd04ff9640 10 client.15597 handle_command_reply: tid=0 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd267de740 1 client.15597 shutdown 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd267de740 1 -- 172.21.15.102:0/1552982394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140eb0230 msgr2=0x559140eb3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd267de740 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140eb0230 0x559140eb3fb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efd14000f80 tx=0x7efd14004c10 comp rx=0 tx=0).stop 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd267de740 1 -- 172.21.15.102:0/1552982394 shutdown_connections 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd267de740 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x559140f27280 0x559140f296a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd267de740 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559140eb0230 0x559140eb3fb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd04ff9640 0 client.15597 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.228+0000 7efd267de740 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559140ea5b60 0x559140eb3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd267de740 1 --2- 172.21.15.102:0/1552982394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559140ea5240 0x559140eb8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd267de740 1 -- 172.21.15.102:0/1552982394 >> 172.21.15.102:0/1552982394 conn(0x559140ea1560 msgr2=0x559140ea0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd267de740 1 -- 172.21.15.102:0/1552982394 shutdown_connections 2024-04-24T23:15:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd267de740 1 -- 172.21.15.102:0/1552982394 wait complete. 2024-04-24T23:15:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd267de740 20 client.15597 trim_cache size 0 max 16384 2024-04-24T23:15:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:15:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:15:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:15:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:15:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd239ee640 1 -- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efd080a2da0 msgr2=0x7efd080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd239ee640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efd080a2da0 0x7efd080a5260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd14022000 tx=0x7efd14023000 comp rx=0 tx=0).stop 2024-04-24T23:15:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd239ee640 1 -- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 msgr2=0x7efd1c198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd239ee640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 0x7efd1c198db0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efd1800d950 tx=0x7efd1800de10 comp rx=0 tx=0).stop 2024-04-24T23:15:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd239ee640 1 -- 172.21.15.102:0/3968294558 shutdown_connections 2024-04-24T23:15:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd239ee640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c106630 0x7efd1c1a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd239ee640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd1c101030 0x7efd1c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd239ee640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efd080a2da0 0x7efd080a5260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.229+0000 7efd239ee640 1 --2- 172.21.15.102:0/3968294558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c100670 0x7efd1c198db0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.230+0000 7efd239ee640 1 -- 172.21.15.102:0/3968294558 >> 172.21.15.102:0/3968294558 conn(0x7efd1c0777c0 msgr2=0x7efd1c103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.230+0000 7efd239ee640 1 -- 172.21.15.102:0/3968294558 shutdown_connections 2024-04-24T23:15:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:29.230+0000 7efd239ee640 1 -- 172.21.15.102:0/3968294558 wait complete. 2024-04-24T23:15:30.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:30 smithi161 ceph-mon[32709]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:15:31.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:30 smithi183 ceph-mon[33137]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:15:31.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:30 smithi102 ceph-mon[29216]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:15:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:32 smithi183 ceph-mon[33137]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:15:33.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:32 smithi102 ceph-mon[29216]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:15:33.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:32 smithi161 ceph-mon[32709]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:15:35.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:34 smithi183 ceph-mon[33137]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:15:35.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:34 smithi102 ceph-mon[29216]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:15:35.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:34 smithi161 ceph-mon[32709]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:15:37.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:36 smithi183 ceph-mon[33137]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:37.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:36 smithi102 ceph-mon[29216]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:37.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:36 smithi161 ceph-mon[32709]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:39.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:38 smithi183 ceph-mon[33137]: pgmap v81: 97 pgs: 97 active+clean; 1.8 GiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:39.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:38 smithi102 ceph-mon[29216]: pgmap v81: 97 pgs: 97 active+clean; 1.8 GiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:39.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:38 smithi161 ceph-mon[32709]: pgmap v81: 97 pgs: 97 active+clean; 1.8 GiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:41.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:40 smithi183 ceph-mon[33137]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:41.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:40 smithi102 ceph-mon[29216]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:41.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:40 smithi161 ceph-mon[32709]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.324+0000 7f625b6e1640 1 -- 172.21.15.102:0/3816109739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254101030 msgr2=0x7f625410d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.324+0000 7f625b6e1640 1 --2- 172.21.15.102:0/3816109739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254101030 0x7f625410d4f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f624c009930 tx=0x7f624c02a420 comp rx=0 tx=0).stop 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.324+0000 7f625b6e1640 1 -- 172.21.15.102:0/3816109739 shutdown_connections 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.324+0000 7f625b6e1640 1 --2- 172.21.15.102:0/3816109739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254101030 0x7f625410d4f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.324+0000 7f625b6e1640 1 --2- 172.21.15.102:0/3816109739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254100670 0x7f6254100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.324+0000 7f625b6e1640 1 --2- 172.21.15.102:0/3816109739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6254106630 0x7f6254106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.324+0000 7f625b6e1640 1 -- 172.21.15.102:0/3816109739 >> 172.21.15.102:0/3816109739 conn(0x7f62540777c0 msgr2=0x7f6254077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.324+0000 7f625b6e1640 1 -- 172.21.15.102:0/3816109739 shutdown_connections 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.324+0000 7f625b6e1640 1 -- 172.21.15.102:0/3816109739 wait complete. 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625b6e1640 1 Processor -- start 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625b6e1640 1 -- start start 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625b6e1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254100670 0x7f6254198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625b6e1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101030 0x7f62541991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625b6e1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6254106630 0x7f62541a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625b6e1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62541108f0 con 0x7f6254100670 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625b6e1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6254110770 con 0x7f6254101030 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625b6e1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6254110a70 con 0x7f6254106630 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625947d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254100670 0x7f6254198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625947d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254100670 0x7f6254198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36802/0 (socket says 172.21.15.102:36802) 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f625947d640 1 -- 172.21.15.102:0/1244384965 learned_addr learned my addr 172.21.15.102:0/1244384965 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f6259c7e640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6254106630 0x7f62541a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.325+0000 7f6258c7c640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101030 0x7f62541991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f625947d640 1 -- 172.21.15.102:0/1244384965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6254106630 msgr2=0x7f62541a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f625947d640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6254106630 0x7f62541a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f625947d640 1 -- 172.21.15.102:0/1244384965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101030 msgr2=0x7f62541991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f625947d640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101030 0x7f62541991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f625947d640 1 -- 172.21.15.102:0/1244384965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62541a4270 con 0x7f6254100670 2024-04-24T23:15:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f625947d640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254100670 0x7f6254198cb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f625000eab0 tx=0x7f625000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f624a7fc640 1 -- 172.21.15.102:0/1244384965 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625000ce40 con 0x7f6254100670 2024-04-24T23:15:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f624a7fc640 1 -- 172.21.15.102:0/1244384965 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6250011920 con 0x7f6254100670 2024-04-24T23:15:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f624a7fc640 1 -- 172.21.15.102:0/1244384965 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250010a00 con 0x7f6254100670 2024-04-24T23:15:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f625b6e1640 1 -- 172.21.15.102:0/1244384965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62541a4440 con 0x7f6254100670 2024-04-24T23:15:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.326+0000 7f625b6e1640 1 -- 172.21.15.102:0/1244384965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62541a4930 con 0x7f6254100670 2024-04-24T23:15:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.329+0000 7f624a7fc640 1 -- 172.21.15.102:0/1244384965 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6250011ad0 con 0x7f6254100670 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.329+0000 7f625e3f4740 1 Processor -- start 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.329+0000 7f625e3f4740 1 -- start start 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.329+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6240 0x55969cef6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6b60 0x55969cf00cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cf03620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f625e3f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55969ce5acb0 con 0x55969cf01230 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f625e3f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55969ce035b0 con 0x55969cef6b60 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f625e3f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55969ce56c70 con 0x55969cef6240 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6258c7c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cf03620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6258c7c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cf03620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36816/0 (socket says 172.21.15.102:36816) 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6258c7c640 1 -- 172.21.15.102:0/813448152 learned_addr learned my addr 172.21.15.102:0/813448152 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f624a7fc640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f62400a2da0 0x7f62400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f625947d640 1 --2- 172.21.15.102:0/813448152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6b60 0x55969cf00cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6259c7e640 1 --2- 172.21.15.102:0/813448152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6240 0x55969cef6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6259c7e640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f62400a2da0 0x7f62400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6258c7c640 1 -- 172.21.15.102:0/813448152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6240 msgr2=0x55969cef6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6258c7c640 1 --2- 172.21.15.102:0/813448152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6240 0x55969cef6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6258c7c640 1 -- 172.21.15.102:0/813448152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6b60 msgr2=0x55969cf00cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6258c7c640 1 --2- 172.21.15.102:0/813448152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6b60 0x55969cf00cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6258c7c640 1 -- 172.21.15.102:0/813448152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55969cb6f210 con 0x55969cf01230 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f624a7fc640 1 -- 172.21.15.102:0/1244384965 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f62500cf940 con 0x7f6254100670 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6258c7c640 1 --2- 172.21.15.102:0/813448152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cf03620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6244009c90 tx=0x7f6244011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3339c17c5cb55b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.330+0000 7f6259c7e640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f62400a2da0 0x7f62400a5260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f624c005ab0 tx=0x7f624c0058d0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f623f7fe640 1 -- 172.21.15.102:0/813448152 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624401e070 con 0x55969cf01230 2024-04-24T23:15:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f623f7fe640 1 -- 172.21.15.102:0/813448152 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6244019ae0 con 0x55969cf01230 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f623f7fe640 1 -- 172.21.15.102:0/813448152 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624400e940 con 0x55969cf01230 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f625e3f4740 1 -- 172.21.15.102:0/813448152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 msgr2=0x55969cf03620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f625e3f4740 1 --2- 172.21.15.102:0/813448152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cf03620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6244009c90 tx=0x7f6244011810 comp rx=0 tx=0).stop 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f625e3f4740 1 -- 172.21.15.102:0/813448152 shutdown_connections 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f625e3f4740 1 --2- 172.21.15.102:0/813448152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cf03620 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f625e3f4740 1 --2- 172.21.15.102:0/813448152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6b60 0x55969cf00cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f625e3f4740 1 --2- 172.21.15.102:0/813448152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6240 0x55969cef6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f625e3f4740 1 -- 172.21.15.102:0/813448152 >> 172.21.15.102:0/813448152 conn(0x55969cef2560 msgr2=0x55969cef2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f625e3f4740 1 -- 172.21.15.102:0/813448152 shutdown_connections 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.331+0000 7f625e3f4740 1 -- 172.21.15.102:0/813448152 wait complete. 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.332+0000 7f625e3f4740 1 -- start start 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.332+0000 7f625e3f4740 1 Processor -- start 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.332+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cefe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.332+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 0x55969cefe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.332+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cefec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.332+0000 7f625e3f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55969cf04020 con 0x55969cf01230 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.332+0000 7f625e3f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55969ce16b50 con 0x55969cef6240 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.332+0000 7f625e3f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55969cf041a0 con 0x55969cef6b60 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f6259c7e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cefec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f625e3f4740 10 client.0 _ll_register_callbacks cb 0x55969ce51ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f6259c7e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cefec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36828/0 (socket says 172.21.15.102:36828) 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f6259c7e640 1 -- 172.21.15.102:0/3059866004 learned_addr learned my addr 172.21.15.102:0/3059866004 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f625947d640 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cefe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f6258c7c640 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 0x55969cefe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f625947d640 1 -- 172.21.15.102:0/3059866004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 msgr2=0x55969cefe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f625947d640 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 0x55969cefe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f625947d640 1 -- 172.21.15.102:0/3059866004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 msgr2=0x55969cefec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f625947d640 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cefec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f625947d640 1 -- 172.21.15.102:0/3059866004 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55969cc3b620 con 0x55969cef6240 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f6258c7c640 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 0x55969cefe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f6259c7e640 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cefec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f625947d640 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cefe1d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f62500cd080 tx=0x7f625009ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d35f6c61ee0ee3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.333+0000 7f623cff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.334+0000 7f623cff9640 1 -- 172.21.15.102:0/3059866004 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625009c550 con 0x55969cef6240 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.334+0000 7f625e3f4740 1 -- 172.21.15.102:0/3059866004 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6244019ae0 con 0x55969cef6240 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.334+0000 7f623cff9640 1 -- 172.21.15.102:0/3059866004 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f62500984c0 con 0x55969cef6240 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.334+0000 7f623cff9640 1 -- 172.21.15.102:0/3059866004 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f625009be50 con 0x55969cef6240 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.334+0000 7f625e3f4740 10 client.25076 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.334+0000 7f625e3f4740 1 -- 172.21.15.102:0/3059866004 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55969cf10270 con 0x55969cef6240 2024-04-24T23:15:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.334+0000 7f623cff9640 1 -- 172.21.15.102:0/3059866004 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6244019ae0 con 0x55969cef6240 2024-04-24T23:15:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.335+0000 7f623cff9640 1 -- 172.21.15.102:0/3059866004 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f625009ece0 con 0x55969cef6240 2024-04-24T23:15:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.335+0000 7f625e3f4740 10 client.25076 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.335+0000 7f625e3f4740 10 client.25076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.335+0000 7f625e3f4740 20 client.25076 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.335+0000 7f625e3f4740 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55969cf116e0 0x55969cf13ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.335+0000 7f625e3f4740 4 client.25076 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.335+0000 7f625e3f4740 1 -- 172.21.15.102:0/3059866004 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55969ce681f0 con 0x55969cf116e0 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.336+0000 7f6258c7c640 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55969cf116e0 0x55969cf13ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.337+0000 7f623cff9640 10 client.25076 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.337+0000 7f6258c7c640 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55969cf116e0 0x55969cf13ad0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20c69a2f1a4f439 server_cookie=798ee0b0c0494b19 in_seq=0 out_seq=0 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.338+0000 7f623cff9640 1 -- 172.21.15.102:0/3059866004 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55969ce681f0 con 0x55969cf116e0 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.338+0000 7f623cff9640 10 client.25076 handle_command_reply: tid=0 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.338+0000 7f625e3f4740 1 client.25076 shutdown 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.338+0000 7f625e3f4740 1 -- 172.21.15.102:0/3059866004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 msgr2=0x55969cefe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.338+0000 7f625e3f4740 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cefe1d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f62500cd080 tx=0x7f625009ba10 comp rx=0 tx=0).stop 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f625e3f4740 1 -- 172.21.15.102:0/3059866004 shutdown_connections 2024-04-24T23:15:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f625e3f4740 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55969cf116e0 0x55969cf13ad0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f625e3f4740 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cefec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f625e3f4740 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 0x55969cefe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f625e3f4740 1 --2- 172.21.15.102:0/3059866004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cefe1d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f625e3f4740 1 -- 172.21.15.102:0/3059866004 >> 172.21.15.102:0/3059866004 conn(0x55969cef2560 msgr2=0x55969cef8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f623cff9640 0 client.25076 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f625e3f4740 1 -- 172.21.15.102:0/3059866004 shutdown_connections 2024-04-24T23:15:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f625e3f4740 1 -- 172.21.15.102:0/3059866004 wait complete. 2024-04-24T23:15:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.339+0000 7f625e3f4740 20 client.25076 trim_cache size 0 max 16384 2024-04-24T23:15:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.356+0000 7f625e3f4740 1 Processor -- start 2024-04-24T23:15:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.356+0000 7f625e3f4740 1 -- start start 2024-04-24T23:15:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.356+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cf00900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.356+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 0x55969cf0cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.356+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cef8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.356+0000 7f625e3f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62541afc30 con 0x55969cf01230 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.356+0000 7f625e3f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62541ad4d0 con 0x55969cef6240 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.356+0000 7f625e3f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62541ab340 con 0x55969cef6b60 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6259c7e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cef8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6259c7e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cef8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36832/0 (socket says 172.21.15.102:36832) 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6259c7e640 1 -- 172.21.15.102:0/2775444166 learned_addr learned my addr 172.21.15.102:0/2775444166 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f625947d640 1 --2- 172.21.15.102:0/2775444166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cf00900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6258c7c640 1 --2- 172.21.15.102:0/2775444166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 0x55969cf0cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6259c7e640 1 -- 172.21.15.102:0/2775444166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 msgr2=0x55969cf0cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6259c7e640 1 --2- 172.21.15.102:0/2775444166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 0x55969cf0cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6259c7e640 1 -- 172.21.15.102:0/2775444166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 msgr2=0x55969cf00900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6259c7e640 1 --2- 172.21.15.102:0/2775444166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cf00900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6259c7e640 1 -- 172.21.15.102:0/2775444166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55969cf03990 con 0x55969cf01230 2024-04-24T23:15:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f6259c7e640 1 --2- 172.21.15.102:0/2775444166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cef8ab0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f624c005940 tx=0x7f624c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd78d367391e3406 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f623effd640 1 -- 172.21.15.102:0/2775444166 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c035040 con 0x55969cf01230 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f623effd640 1 -- 172.21.15.102:0/2775444166 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f624c043070 con 0x55969cf01230 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.357+0000 7f623effd640 1 -- 172.21.15.102:0/2775444166 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c004ac0 con 0x55969cf01230 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 -- 172.21.15.102:0/2775444166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 msgr2=0x55969cef8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 --2- 172.21.15.102:0/2775444166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cef8ab0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f624c005940 tx=0x7f624c005710 comp rx=0 tx=0).stop 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 -- 172.21.15.102:0/2775444166 shutdown_connections 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 --2- 172.21.15.102:0/2775444166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cf01230 0x55969cef8ab0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 --2- 172.21.15.102:0/2775444166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cef6b60 0x55969cf0cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 --2- 172.21.15.102:0/2775444166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cf00900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 -- 172.21.15.102:0/2775444166 >> 172.21.15.102:0/2775444166 conn(0x55969cef2560 msgr2=0x55969cef9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 -- 172.21.15.102:0/2775444166 shutdown_connections 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 -- 172.21.15.102:0/2775444166 wait complete. 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.358+0000 7f625e3f4740 1 -- start start 2024-04-24T23:15:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f625e3f4740 1 Processor -- start 2024-04-24T23:15:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cf093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cef6b60 0x55969cf09930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f625e3f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cf01230 0x55969cefa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f625e3f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62541ab990 con 0x55969cef6b60 2024-04-24T23:15:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f625e3f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62541ace00 con 0x55969cef6240 2024-04-24T23:15:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f625e3f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62541ac060 con 0x55969cf01230 2024-04-24T23:15:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f625e3f4740 10 client.0 _ll_register_callbacks cb 0x7f625000ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f6258c7c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cef6b60 0x55969cf09930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f6258c7c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cef6b60 0x55969cf09930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36844/0 (socket says 172.21.15.102:36844) 2024-04-24T23:15:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f6258c7c640 1 -- 172.21.15.102:0/418909928 learned_addr learned my addr 172.21.15.102:0/418909928 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f6259c7e640 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cf01230 0x55969cefa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f625947d640 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cf093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f6258c7c640 1 -- 172.21.15.102:0/418909928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cf01230 msgr2=0x55969cefa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f6258c7c640 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cf01230 0x55969cefa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f6258c7c640 1 -- 172.21.15.102:0/418909928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 msgr2=0x55969cf093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.359+0000 7f6258c7c640 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cf093f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f6258c7c640 1 -- 172.21.15.102:0/418909928 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55969cefaa00 con 0x55969cef6b60 2024-04-24T23:15:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f6259c7e640 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cf01230 0x55969cefa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f6258c7c640 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cef6b60 0x55969cf09930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6244009a50 tx=0x7f624400e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b87c3130e34acb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f623cff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:15:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f623cff9640 1 -- 172.21.15.102:0/418909928 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624401e040 con 0x55969cef6b60 2024-04-24T23:15:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f623cff9640 1 -- 172.21.15.102:0/418909928 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f62440052b0 con 0x55969cef6b60 2024-04-24T23:15:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f625e3f4740 1 -- 172.21.15.102:0/418909928 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f624c043070 con 0x55969cef6b60 2024-04-24T23:15:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f623cff9640 1 -- 172.21.15.102:0/418909928 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6244039050 con 0x55969cef6b60 2024-04-24T23:15:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f625e3f4740 10 client.15615 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f625e3f4740 1 -- 172.21.15.102:0/418909928 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55969cf77500 con 0x55969cef6b60 2024-04-24T23:15:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f623cff9640 1 -- 172.21.15.102:0/418909928 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f624c043070 con 0x55969cef6b60 2024-04-24T23:15:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f623cff9640 1 -- 172.21.15.102:0/418909928 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f624402b7a0 con 0x55969cef6b60 2024-04-24T23:15:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f625e3f4740 10 client.15615 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f625e3f4740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.360+0000 7f625e3f4740 20 client.15615 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.361+0000 7f625e3f4740 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55969cf77bc0 0x55969cf79fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.361+0000 7f625e3f4740 4 client.15615 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:15:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.361+0000 7f625e3f4740 1 -- 172.21.15.102:0/418909928 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55969ce62540 con 0x55969cf77bc0 2024-04-24T23:15:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.361+0000 7f625947d640 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55969cf77bc0 0x55969cf79fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.361+0000 7f623cff9640 10 client.15615 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.361+0000 7f625947d640 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55969cf77bc0 0x55969cf79fe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ef5d048bcd8838 server_cookie=d53f2913764a455d in_seq=0 out_seq=0 2024-04-24T23:15:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f623cff9640 1 -- 172.21.15.102:0/418909928 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55969ce62540 con 0x55969cf77bc0 2024-04-24T23:15:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f623cff9640 10 client.15615 handle_command_reply: tid=0 2024-04-24T23:15:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 client.15615 shutdown 2024-04-24T23:15:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 -- 172.21.15.102:0/418909928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cef6b60 msgr2=0x55969cf09930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cef6b60 0x55969cf09930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6244009a50 tx=0x7f624400e4c0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 -- 172.21.15.102:0/418909928 shutdown_connections 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55969cf01230 0x55969cefa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55969cf77bc0 0x55969cf79fe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55969cef6b60 0x55969cf09930 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 --2- 172.21.15.102:0/418909928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55969cef6240 0x55969cf093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f623cff9640 0 client.15615 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 -- 172.21.15.102:0/418909928 >> 172.21.15.102:0/418909928 conn(0x55969cef2560 msgr2=0x55969cef0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 -- 172.21.15.102:0/418909928 shutdown_connections 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 1 -- 172.21.15.102:0/418909928 wait complete. 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.369+0000 7f625e3f4740 20 client.15615 trim_cache size 0 max 16384 2024-04-24T23:15:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:15:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:15:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:15:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 -- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f62400a2da0 msgr2=0x7f62400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f62400a2da0 0x7f62400a5260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f624c005ab0 tx=0x7f624c0058d0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 -- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254100670 msgr2=0x7f6254198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254100670 0x7f6254198cb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f625000eab0 tx=0x7f625000ef70 comp rx=0 tx=0).stop 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 -- 172.21.15.102:0/1244384965 shutdown_connections 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6254106630 0x7f62541a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101030 0x7f62541991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f62400a2da0 0x7f62400a5260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 --2- 172.21.15.102:0/1244384965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6254100670 0x7f6254198cb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 -- 172.21.15.102:0/1244384965 >> 172.21.15.102:0/1244384965 conn(0x7f62540777c0 msgr2=0x7f6254103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 -- 172.21.15.102:0/1244384965 shutdown_connections 2024-04-24T23:15:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:42.370+0000 7f625b6e1640 1 -- 172.21.15.102:0/1244384965 wait complete. 2024-04-24T23:15:43.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:42 smithi183 ceph-mon[33137]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:42 smithi102 ceph-mon[29216]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:43.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:42 smithi161 ceph-mon[32709]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:44 smithi183 ceph-mon[33137]: pgmap v84: 97 pgs: 97 active+clean; 2.0 GiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 27 op/s 2024-04-24T23:15:45.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:44 smithi102 ceph-mon[29216]: pgmap v84: 97 pgs: 97 active+clean; 2.0 GiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 27 op/s 2024-04-24T23:15:45.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:44 smithi161 ceph-mon[32709]: pgmap v84: 97 pgs: 97 active+clean; 2.0 GiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 27 op/s 2024-04-24T23:15:47.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:46 smithi183 ceph-mon[33137]: pgmap v85: 97 pgs: 97 active+clean; 2.0 GiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:47.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:46 smithi102 ceph-mon[29216]: pgmap v85: 97 pgs: 97 active+clean; 2.0 GiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:47.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:46 smithi161 ceph-mon[32709]: pgmap v85: 97 pgs: 97 active+clean; 2.0 GiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:49.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:48 smithi183 ceph-mon[33137]: pgmap v86: 97 pgs: 97 active+clean; 2.1 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:49.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:48 smithi102 ceph-mon[29216]: pgmap v86: 97 pgs: 97 active+clean; 2.1 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:49.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:48 smithi161 ceph-mon[32709]: pgmap v86: 97 pgs: 97 active+clean; 2.1 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:51.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:50 smithi183 ceph-mon[33137]: pgmap v87: 97 pgs: 97 active+clean; 2.1 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:51.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:50 smithi102 ceph-mon[29216]: pgmap v87: 97 pgs: 97 active+clean; 2.1 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:50 smithi161 ceph-mon[32709]: pgmap v87: 97 pgs: 97 active+clean; 2.1 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:53.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:52 smithi183 ceph-mon[33137]: pgmap v88: 97 pgs: 97 active+clean; 2.1 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:53.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:52 smithi102 ceph-mon[29216]: pgmap v88: 97 pgs: 97 active+clean; 2.1 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:53.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:52 smithi161 ceph-mon[32709]: pgmap v88: 97 pgs: 97 active+clean; 2.1 GiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:15:55.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:54 smithi183 ceph-mon[33137]: pgmap v89: 97 pgs: 97 active+clean; 2.3 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:15:55.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:54 smithi102 ceph-mon[29216]: pgmap v89: 97 pgs: 97 active+clean; 2.3 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:15:55.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:54 smithi161 ceph-mon[32709]: pgmap v89: 97 pgs: 97 active+clean; 2.3 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:15:55.472 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.469+0000 7fed11909640 1 -- 172.21.15.102:0/1752862261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c106630 msgr2=0x7fed0c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.469+0000 7fed11909640 1 --2- 172.21.15.102:0/1752862261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c106630 0x7fed0c106a10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009930 tx=0x7fecfc02a420 comp rx=0 tx=0).stop 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.469+0000 7fed11909640 1 -- 172.21.15.102:0/1752862261 shutdown_connections 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.469+0000 7fed11909640 1 --2- 172.21.15.102:0/1752862261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0c101030 0x7fed0c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.469+0000 7fed11909640 1 --2- 172.21.15.102:0/1752862261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0c100670 0x7fed0c100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.469+0000 7fed11909640 1 --2- 172.21.15.102:0/1752862261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c106630 0x7fed0c106a10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.469+0000 7fed11909640 1 -- 172.21.15.102:0/1752862261 >> 172.21.15.102:0/1752862261 conn(0x7fed0c0777c0 msgr2=0x7fed0c077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.469+0000 7fed11909640 1 -- 172.21.15.102:0/1752862261 shutdown_connections 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.469+0000 7fed11909640 1 -- 172.21.15.102:0/1752862261 wait complete. 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed11909640 1 Processor -- start 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed11909640 1 -- start start 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed11909640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c100670 0x7fed0c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed11909640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0c101030 0x7fed0c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed11909640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0c106630 0x7fed0c19b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed11909640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1108b0 con 0x7fed0c100670 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed11909640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c110730 con 0x7fed0c101030 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed11909640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c110a30 con 0x7fed0c106630 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed0affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c100670 0x7fed0c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed0affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c100670 0x7fed0c196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40790/0 (socket says 172.21.15.102:40790) 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed0affd640 1 -- 172.21.15.102:0/625957674 learned_addr learned my addr 172.21.15.102:0/625957674 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed0a7fc640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0c101030 0x7fed0c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.470+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0c106630 0x7fed0c19b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed0affd640 1 -- 172.21.15.102:0/625957674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0c106630 msgr2=0x7fed0c19b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed0affd640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0c106630 0x7fed0c19b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed0affd640 1 -- 172.21.15.102:0/625957674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0c101030 msgr2=0x7fed0c197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed0affd640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0c101030 0x7fed0c197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed0affd640 1 -- 172.21.15.102:0/625957674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c19bd40 con 0x7fed0c100670 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0c106630 0x7fed0c19b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed0affd640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c100670 0x7fed0c196d50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fecfc004770 tx=0x7fecfc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed10907640 1 -- 172.21.15.102:0/625957674 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc03f070 con 0x7fed0c100670 2024-04-24T23:15:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed10907640 1 -- 172.21.15.102:0/625957674 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecfc03b460 con 0x7fed0c100670 2024-04-24T23:15:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed11909640 1 -- 172.21.15.102:0/625957674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c19bfd0 con 0x7fed0c100670 2024-04-24T23:15:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed10907640 1 -- 172.21.15.102:0/625957674 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc035070 con 0x7fed0c100670 2024-04-24T23:15:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.471+0000 7fed11909640 1 -- 172.21.15.102:0/625957674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c1a38c0 con 0x7fed0c100670 2024-04-24T23:15:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.473+0000 7fed10907640 1 -- 172.21.15.102:0/625957674 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fecfc03b610 con 0x7fed0c100670 2024-04-24T23:15:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.473+0000 7fed14614740 1 Processor -- start 2024-04-24T23:15:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed14614740 1 -- start start 2024-04-24T23:15:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed14614740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 0x555c8d55d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed14614740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d55db60 0x555c8d567cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed14614740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d568230 0x555c8d56a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed14614740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555c8d4c1cb0 con 0x555c8d55db60 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed14614740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c8d46a5b0 con 0x555c8d55d240 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed14614740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c8d4bdc70 con 0x555c8d568230 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d55db60 0x555c8d567cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d55db60 0x555c8d567cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40794/0 (socket says 172.21.15.102:40794) 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0affd640 1 -- 172.21.15.102:0/3204574739 learned_addr learned my addr 172.21.15.102:0/3204574739 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 0x555c8d55d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed10907640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fecf40a2da0 0x7fecf40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0a7fc640 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d568230 0x555c8d56a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0affd640 1 -- 172.21.15.102:0/3204574739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d568230 msgr2=0x555c8d56a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0affd640 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d568230 0x555c8d56a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0affd640 1 -- 172.21.15.102:0/3204574739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 msgr2=0x555c8d55d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0affd640 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 0x555c8d55d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0affd640 1 -- 172.21.15.102:0/3204574739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8d1d6210 con 0x555c8d55db60 2024-04-24T23:15:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0a7fc640 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d568230 0x555c8d56a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 0x555c8d55d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fecf40a2da0 0x7fecf40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.474+0000 7fed10907640 1 -- 172.21.15.102:0/625957674 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fecfc0ef7e0 con 0x7fed0c100670 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.475+0000 7fed0affd640 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d55db60 0x555c8d567cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0ece90 tx=0x7fecfc0b1180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7eb98f2d961ab27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.475+0000 7fecf1ffb640 1 -- 172.21.15.102:0/3204574739 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0bb830 con 0x555c8d55db60 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.475+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fecf40a2da0 0x7fecf40a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fed04005fd0 tx=0x7fed04004a40 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.475+0000 7fecf1ffb640 1 -- 172.21.15.102:0/3204574739 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecfc086d80 con 0x555c8d55db60 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.475+0000 7fecf1ffb640 1 -- 172.21.15.102:0/3204574739 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc08bbc0 con 0x555c8d55db60 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.475+0000 7fed14614740 1 -- 172.21.15.102:0/3204574739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d55db60 msgr2=0x555c8d567cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.475+0000 7fed14614740 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d55db60 0x555c8d567cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0ece90 tx=0x7fecfc0b1180 comp rx=0 tx=0).stop 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 -- 172.21.15.102:0/3204574739 shutdown_connections 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d568230 0x555c8d56a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d55db60 0x555c8d567cf0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 --2- 172.21.15.102:0/3204574739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 0x555c8d55d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 -- 172.21.15.102:0/3204574739 >> 172.21.15.102:0/3204574739 conn(0x555c8d559560 msgr2=0x555c8d559940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 -- 172.21.15.102:0/3204574739 shutdown_connections 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 -- 172.21.15.102:0/3204574739 wait complete. 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 -- start start 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 Processor -- start 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 0x555c8d565160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 0x555c8d5656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.476+0000 7fed14614740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d565be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed14614740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555c8d56ade0 con 0x555c8d568230 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed14614740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c8d47db50 con 0x555c8d55db60 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed14614740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c8d56af60 con 0x555c8d55d240 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed14614740 10 client.0 _ll_register_callbacks cb 0x555c8d4b8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d565be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d565be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40796/0 (socket says 172.21.15.102:40796) 2024-04-24T23:15:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0b7fe640 1 -- 172.21.15.102:0/1688662292 learned_addr learned my addr 172.21.15.102:0/1688662292 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0a7fc640 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 0x555c8d5656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0affd640 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 0x555c8d565160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0b7fe640 1 -- 172.21.15.102:0/1688662292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 msgr2=0x555c8d565160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 0x555c8d565160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0b7fe640 1 -- 172.21.15.102:0/1688662292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 msgr2=0x555c8d5656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 0x555c8d5656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0b7fe640 1 -- 172.21.15.102:0/1688662292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8d2a2620 con 0x555c8d568230 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d565be0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fed04005c00 tx=0x7fed04004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cbb6b22d603a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7feceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7feceaffd640 1 -- 172.21.15.102:0/1688662292 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0401d7a0 con 0x555c8d568230 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7feceaffd640 1 -- 172.21.15.102:0/1688662292 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fed0401d940 con 0x555c8d568230 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7fed14614740 1 -- 172.21.15.102:0/1688662292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecfc086d80 con 0x555c8d568230 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.477+0000 7feceaffd640 1 -- 172.21.15.102:0/1688662292 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fed0401db40 con 0x555c8d568230 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7fed14614740 10 client.15627 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7fed14614740 1 -- 172.21.15.102:0/1688662292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555c8d576e80 con 0x555c8d568230 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7feceaffd640 1 -- 172.21.15.102:0/1688662292 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecfc086d80 con 0x555c8d568230 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7feceaffd640 1 -- 172.21.15.102:0/1688662292 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7fed04011430 con 0x555c8d568230 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7fed14614740 10 client.15627 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7fed14614740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7fed14614740 20 client.15627 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7fed14614740 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555c8d5784a0 0x555c8d57a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7fed14614740 4 client.15627 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7fed14614740 1 -- 172.21.15.102:0/1688662292 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c8d4cf1f0 con 0x555c8d5784a0 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.478+0000 7fed0affd640 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555c8d5784a0 0x555c8d57a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.479+0000 7feceaffd640 10 client.15627 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.479+0000 7fed0affd640 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555c8d5784a0 0x555c8d57a8b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1770752552756db server_cookie=63cb2ad84fa8990b in_seq=0 out_seq=0 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.479+0000 7feceaffd640 1 -- 172.21.15.102:0/1688662292 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c8d4cf1f0 con 0x555c8d5784a0 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.479+0000 7feceaffd640 10 client.15627 handle_command_reply: tid=0 2024-04-24T23:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.479+0000 7fed14614740 1 client.15627 shutdown 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 -- 172.21.15.102:0/1688662292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 msgr2=0x555c8d565be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d565be0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fed04005c00 tx=0x7fed04004ab0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 -- 172.21.15.102:0/1688662292 shutdown_connections 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555c8d5784a0 0x555c8d57a8b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d565be0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 0x555c8d5656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 --2- 172.21.15.102:0/1688662292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 0x555c8d565160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7feceaffd640 0 client.15627 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 -- 172.21.15.102:0/1688662292 >> 172.21.15.102:0/1688662292 conn(0x555c8d559560 msgr2=0x555c8d55f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 -- 172.21.15.102:0/1688662292 shutdown_connections 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 1 -- 172.21.15.102:0/1688662292 wait complete. 2024-04-24T23:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.480+0000 7fed14614740 20 client.15627 trim_cache size 0 max 16384 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed14614740 1 Processor -- start 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed14614740 1 -- start start 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed14614740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 0x555c8d573c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed14614740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55db60 0x555c8d558d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed14614740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d5801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed14614740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1ada10 con 0x555c8d568230 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed14614740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1ab2b0 con 0x555c8d55d240 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed14614740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1a9120 con 0x555c8d55db60 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d5801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d5801c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40812/0 (socket says 172.21.15.102:40812) 2024-04-24T23:15:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0b7fe640 1 -- 172.21.15.102:0/3632750657 learned_addr learned my addr 172.21.15.102:0/3632750657 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0affd640 1 --2- 172.21.15.102:0/3632750657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 0x555c8d573c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0b7fe640 1 -- 172.21.15.102:0/3632750657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55db60 msgr2=0x555c8d558d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/3632750657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55db60 0x555c8d558d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0b7fe640 1 -- 172.21.15.102:0/3632750657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 msgr2=0x555c8d573c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/3632750657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 0x555c8d573c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0b7fe640 1 -- 172.21.15.102:0/3632750657 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8d5636b0 con 0x555c8d568230 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.496+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/3632750657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d5801c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fed04011820 tx=0x7fed04000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fa8c42fe352436 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fecf17fa640 1 -- 172.21.15.102:0/3632750657 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0401dce0 con 0x555c8d568230 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fecf17fa640 1 -- 172.21.15.102:0/3632750657 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed0401de80 con 0x555c8d568230 2024-04-24T23:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fecf17fa640 1 -- 172.21.15.102:0/3632750657 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04011dc0 con 0x555c8d568230 2024-04-24T23:15:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fed14614740 1 -- 172.21.15.102:0/3632750657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 msgr2=0x555c8d5801c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fed14614740 1 --2- 172.21.15.102:0/3632750657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d5801c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fed04011820 tx=0x7fed04000f40 comp rx=0 tx=0).stop 2024-04-24T23:15:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fed14614740 1 -- 172.21.15.102:0/3632750657 shutdown_connections 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fed14614740 1 --2- 172.21.15.102:0/3632750657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d5801c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fed14614740 1 --2- 172.21.15.102:0/3632750657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55db60 0x555c8d558d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fed14614740 1 --2- 172.21.15.102:0/3632750657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55d240 0x555c8d573c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fed14614740 1 -- 172.21.15.102:0/3632750657 >> 172.21.15.102:0/3632750657 conn(0x555c8d559560 msgr2=0x555c8d55f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fed14614740 1 -- 172.21.15.102:0/3632750657 shutdown_connections 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.497+0000 7fed14614740 1 -- 172.21.15.102:0/3632750657 wait complete. 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed14614740 1 -- start start 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed14614740 1 Processor -- start 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed14614740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 0x555c8d570320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed14614740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 0x555c8d5619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed14614740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d561f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed14614740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1a9770 con 0x555c8d568230 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed14614740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1aabe0 con 0x555c8d55db60 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed14614740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1a9e40 con 0x555c8d55d240 2024-04-24T23:15:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed14614740 10 client.0 _ll_register_callbacks cb 0x7fed04009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed0b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d561f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed0b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d561f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40826/0 (socket says 172.21.15.102:40826) 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed0b7fe640 1 -- 172.21.15.102:0/269494153 learned_addr learned my addr 172.21.15.102:0/269494153 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.498+0000 7fed0affd640 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 0x555c8d570320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed0a7fc640 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 0x555c8d5619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed0b7fe640 1 -- 172.21.15.102:0/269494153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 msgr2=0x555c8d570320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 0x555c8d570320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed0b7fe640 1 -- 172.21.15.102:0/269494153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 msgr2=0x555c8d5619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 0x555c8d5619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed0b7fe640 1 -- 172.21.15.102:0/269494153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8d562550 con 0x555c8d568230 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed0b7fe640 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d561f10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fed04012f50 tx=0x7fed0401dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1656ef02f541c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7feceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7feceaffd640 1 -- 172.21.15.102:0/269494153 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0401dce0 con 0x555c8d568230 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7feceaffd640 1 -- 172.21.15.102:0/269494153 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fed04011c10 con 0x555c8d568230 2024-04-24T23:15:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed14614740 1 -- 172.21.15.102:0/269494153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c8d4c9540 con 0x555c8d568230 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7feceaffd640 1 -- 172.21.15.102:0/269494153 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fed04011e10 con 0x555c8d568230 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed14614740 10 client.15633 fetch_fsmap learned FSMap version 23 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.499+0000 7fed14614740 1 -- 172.21.15.102:0/269494153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555c8d5de870 con 0x555c8d568230 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7feceaffd640 1 -- 172.21.15.102:0/269494153 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555c8d4c9540 con 0x555c8d568230 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7feceaffd640 1 -- 172.21.15.102:0/269494153 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7fed04022470 con 0x555c8d568230 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7fed14614740 10 client.15633 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7fed14614740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7fed14614740 20 client.15633 populate_metadata read hostname 'smithi102' 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7fed14614740 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555c8d5def40 0x555c8d5e1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7fed14614740 4 client.15633 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7fed14614740 1 -- 172.21.15.102:0/269494153 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c8d4478e0 con 0x555c8d5def40 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7fed0affd640 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555c8d5def40 0x555c8d5e1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7fed0affd640 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555c8d5def40 0x555c8d5e1360 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7cd517e1210c854 server_cookie=359b3492eac525c7 in_seq=0 out_seq=0 2024-04-24T23:15:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.500+0000 7feceaffd640 10 client.15633 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7feceaffd640 1 -- 172.21.15.102:0/269494153 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c8d4478e0 con 0x555c8d5def40 2024-04-24T23:15:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7feceaffd640 10 client.15633 handle_command_reply: tid=0 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 client.15633 shutdown 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 -- 172.21.15.102:0/269494153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 msgr2=0x555c8d561f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d561f10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fed04012f50 tx=0x7fed0401dad0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 -- 172.21.15.102:0/269494153 shutdown_connections 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555c8d5def40 0x555c8d5e1360 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c8d568230 0x555c8d561f10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c8d55db60 0x555c8d5619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 --2- 172.21.15.102:0/269494153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c8d55d240 0x555c8d570320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7feceaffd640 0 client.15633 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 -- 172.21.15.102:0/269494153 >> 172.21.15.102:0/269494153 conn(0x555c8d559560 msgr2=0x555c8d565120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 -- 172.21.15.102:0/269494153 shutdown_connections 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.508+0000 7fed14614740 1 -- 172.21.15.102:0/269494153 wait complete. 2024-04-24T23:15:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed14614740 20 client.15633 trim_cache size 0 max 16384 2024-04-24T23:15:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:15:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:15:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:15:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:15:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 -- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fecf40a2da0 msgr2=0x7fecf40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fecf40a2da0 0x7fecf40a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fed04005fd0 tx=0x7fed04004a40 comp rx=0 tx=0).stop 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 -- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c100670 msgr2=0x7fed0c196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c100670 0x7fed0c196d50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fecfc004770 tx=0x7fecfc0042f0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 -- 172.21.15.102:0/625957674 shutdown_connections 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed0c106630 0x7fed0c19b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0c101030 0x7fed0c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fecf40a2da0 0x7fecf40a5260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 --2- 172.21.15.102:0/625957674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed0c100670 0x7fed0c196d50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 -- 172.21.15.102:0/625957674 >> 172.21.15.102:0/625957674 conn(0x7fed0c0777c0 msgr2=0x7fed0c1031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 -- 172.21.15.102:0/625957674 shutdown_connections 2024-04-24T23:15:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:15:55.509+0000 7fed11909640 1 -- 172.21.15.102:0/625957674 wait complete. 2024-04-24T23:15:57.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:56 smithi183 ceph-mon[33137]: pgmap v90: 97 pgs: 97 active+clean; 2.3 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:15:57.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:56 smithi102 ceph-mon[29216]: pgmap v90: 97 pgs: 97 active+clean; 2.3 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:15:57.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:56 smithi161 ceph-mon[32709]: pgmap v90: 97 pgs: 97 active+clean; 2.3 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:15:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:15:58 smithi183 ceph-mon[33137]: pgmap v91: 97 pgs: 97 active+clean; 2.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:15:58 smithi102 ceph-mon[29216]: pgmap v91: 97 pgs: 97 active+clean; 2.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:15:59.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:15:58 smithi161 ceph-mon[32709]: pgmap v91: 97 pgs: 97 active+clean; 2.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:01.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:00 smithi183 ceph-mon[33137]: pgmap v92: 97 pgs: 97 active+clean; 2.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:01.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:00 smithi102 ceph-mon[29216]: pgmap v92: 97 pgs: 97 active+clean; 2.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:01.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:00 smithi161 ceph-mon[32709]: pgmap v92: 97 pgs: 97 active+clean; 2.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:03.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:02 smithi183 ceph-mon[33137]: pgmap v93: 97 pgs: 97 active+clean; 2.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:03.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:02 smithi102 ceph-mon[29216]: pgmap v93: 97 pgs: 97 active+clean; 2.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:03.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:02 smithi161 ceph-mon[32709]: pgmap v93: 97 pgs: 97 active+clean; 2.4 GiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:05.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:04 smithi183 ceph-mon[33137]: pgmap v94: 97 pgs: 97 active+clean; 2.6 GiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-04-24T23:16:05.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:04 smithi102 ceph-mon[29216]: pgmap v94: 97 pgs: 97 active+clean; 2.6 GiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-04-24T23:16:05.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:04 smithi161 ceph-mon[32709]: pgmap v94: 97 pgs: 97 active+clean; 2.6 GiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-04-24T23:16:07.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:06 smithi183 ceph-mon[33137]: pgmap v95: 97 pgs: 97 active+clean; 2.6 GiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:16:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:06 smithi102 ceph-mon[29216]: pgmap v95: 97 pgs: 97 active+clean; 2.6 GiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:16:07.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:06 smithi161 ceph-mon[32709]: pgmap v95: 97 pgs: 97 active+clean; 2.6 GiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:16:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.598+0000 7f57084a7640 1 -- 172.21.15.102:0/2839496868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700100670 msgr2=0x7f5700100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.598+0000 7f57084a7640 1 --2- 172.21.15.102:0/2839496868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700100670 0x7f5700100af0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009930 tx=0x7f56f002a420 comp rx=0 tx=0).stop 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.598+0000 7f57084a7640 1 -- 172.21.15.102:0/2839496868 shutdown_connections 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.598+0000 7f57084a7640 1 --2- 172.21.15.102:0/2839496868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5700101030 0x7f570010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.598+0000 7f57084a7640 1 --2- 172.21.15.102:0/2839496868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700100670 0x7f5700100af0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.598+0000 7f57084a7640 1 --2- 172.21.15.102:0/2839496868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5700106630 0x7f5700106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.598+0000 7f57084a7640 1 -- 172.21.15.102:0/2839496868 >> 172.21.15.102:0/2839496868 conn(0x7f57000777c0 msgr2=0x7f5700077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.598+0000 7f57084a7640 1 -- 172.21.15.102:0/2839496868 shutdown_connections 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.598+0000 7f57084a7640 1 -- 172.21.15.102:0/2839496868 wait complete. 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f57084a7640 1 Processor -- start 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f57084a7640 1 -- start start 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f57084a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5700100670 0x7f5700197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f57084a7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700101030 0x7f57001984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f57084a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5700106630 0x7f570019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f57084a7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5700110940 con 0x7f5700101030 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f57084a7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57001107c0 con 0x7f5700100670 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f57084a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5700110ac0 con 0x7f5700106630 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f5705a42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700101030 0x7f57001984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.599+0000 7f5705a42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700101030 0x7f57001984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37932/0 (socket says 172.21.15.102:37932) 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f5705a42640 1 -- 172.21.15.102:0/1085057336 learned_addr learned my addr 172.21.15.102:0/1085057336 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f5706a44640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5700106630 0x7f570019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f5706243640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5700100670 0x7f5700197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f5705a42640 1 -- 172.21.15.102:0/1085057336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5700106630 msgr2=0x7f570019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f5705a42640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5700106630 0x7f570019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f5705a42640 1 -- 172.21.15.102:0/1085057336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5700100670 msgr2=0x7f5700197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f5705a42640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5700100670 0x7f5700197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f5705a42640 1 -- 172.21.15.102:0/1085057336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570019cfa0 con 0x7f5700101030 2024-04-24T23:16:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f5705a42640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700101030 0x7f57001984f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f56f0005ab0 tx=0x7f56f0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f56f77fe640 1 -- 172.21.15.102:0/1085057336 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f003f070 con 0x7f5700101030 2024-04-24T23:16:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f57084a7640 1 -- 172.21.15.102:0/1085057336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570019d230 con 0x7f5700101030 2024-04-24T23:16:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f56f77fe640 1 -- 172.21.15.102:0/1085057336 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56f0031d60 con 0x7f5700101030 2024-04-24T23:16:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f56f77fe640 1 -- 172.21.15.102:0/1085057336 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0035030 con 0x7f5700101030 2024-04-24T23:16:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.600+0000 7f57084a7640 1 -- 172.21.15.102:0/1085057336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57001a4b20 con 0x7f5700101030 2024-04-24T23:16:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.603+0000 7f56f77fe640 1 -- 172.21.15.102:0/1085057336 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f56f0042070 con 0x7f5700101030 2024-04-24T23:16:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.603+0000 7f570b1b0740 1 Processor -- start 2024-04-24T23:16:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f570b1b0740 1 -- start start 2024-04-24T23:16:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 0x563999474a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474f70 0x56399947f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x563999481a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f570b1b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639993d4cb0 con 0x56399947f640 2024-04-24T23:16:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f570b1b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56399937d5b0 con 0x563999474f70 2024-04-24T23:16:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f570b1b0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639993d0c70 con 0x563999474650 2024-04-24T23:16:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f5705a42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x563999481a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f5705a42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x563999481a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37942/0 (socket says 172.21.15.102:37942) 2024-04-24T23:16:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f5705a42640 1 -- 172.21.15.102:0/2890904235 learned_addr learned my addr 172.21.15.102:0/2890904235 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f5706243640 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474f70 0x56399947f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f5706a44640 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 0x563999474a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f56f77fe640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f56ec0a2da0 0x7f56ec0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f5705a42640 1 -- 172.21.15.102:0/2890904235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 msgr2=0x563999474a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f5705a42640 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 0x563999474a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.604+0000 7f5705a42640 1 -- 172.21.15.102:0/2890904235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474f70 msgr2=0x56399947f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f5705a42640 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474f70 0x56399947f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f5705a42640 1 -- 172.21.15.102:0/2890904235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639990e9210 con 0x56399947f640 2024-04-24T23:16:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f5706a44640 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 0x563999474a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f5706243640 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474f70 0x56399947f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f5706a44640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f56ec0a2da0 0x7f56ec0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f56f77fe640 1 -- 172.21.15.102:0/1085057336 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f56f00ef910 con 0x7f5700101030 2024-04-24T23:16:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f5705a42640 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x563999481a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f56f00ecfc0 tx=0x7f56f00414b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498b849b2730f7d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f56f4bfe640 1 -- 172.21.15.102:0/2890904235 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00bb780 con 0x56399947f640 2024-04-24T23:16:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f56f4bfe640 1 -- 172.21.15.102:0/2890904235 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56f00bc520 con 0x56399947f640 2024-04-24T23:16:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f56f4bfe640 1 -- 172.21.15.102:0/2890904235 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00bbe20 con 0x56399947f640 2024-04-24T23:16:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.605+0000 7f5706a44640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f56ec0a2da0 0x7f56ec0a5260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56f80098b0 tx=0x7f56f8004a40 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 -- 172.21.15.102:0/2890904235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 msgr2=0x563999481a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x563999481a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f56f00ecfc0 tx=0x7f56f00414b0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 -- 172.21.15.102:0/2890904235 shutdown_connections 2024-04-24T23:16:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x563999481a30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474f70 0x56399947f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 --2- 172.21.15.102:0/2890904235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 0x563999474a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 -- 172.21.15.102:0/2890904235 >> 172.21.15.102:0/2890904235 conn(0x563999470970 msgr2=0x563999470d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 -- 172.21.15.102:0/2890904235 shutdown_connections 2024-04-24T23:16:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 -- 172.21.15.102:0/2890904235 wait complete. 2024-04-24T23:16:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 -- start start 2024-04-24T23:16:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.606+0000 7f570b1b0740 1 Processor -- start 2024-04-24T23:16:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 0x56399947c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474f70 0x56399947ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x56399947cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f570b1b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639994821f0 con 0x56399947f640 2024-04-24T23:16:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f570b1b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563999390b50 con 0x563999474650 2024-04-24T23:16:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f570b1b0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563999482370 con 0x563999474f70 2024-04-24T23:16:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706a44640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x56399947cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706a44640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x56399947cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37946/0 (socket says 172.21.15.102:37946) 2024-04-24T23:16:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706a44640 1 -- 172.21.15.102:0/3935212035 learned_addr learned my addr 172.21.15.102:0/3935212035 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f570b1b0740 10 client.0 _ll_register_callbacks cb 0x5639993cbca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706243640 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 0x56399947c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5705a42640 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474f70 0x56399947ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706a44640 1 -- 172.21.15.102:0/3935212035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474f70 msgr2=0x56399947ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706a44640 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474f70 0x56399947ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706a44640 1 -- 172.21.15.102:0/3935212035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 msgr2=0x56399947c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706a44640 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 0x56399947c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706a44640 1 -- 172.21.15.102:0/3935212035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639991b5620 con 0x56399947f640 2024-04-24T23:16:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5705a42640 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474f70 0x56399947ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f5706a44640 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x56399947cfa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f56f8013960 tx=0x7f56f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db47d70f25f82f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f56e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:16:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3935212035 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f801ed70 con 0x56399947f640 2024-04-24T23:16:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3935212035 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f56f8028470 con 0x56399947f640 2024-04-24T23:16:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f570b1b0740 1 -- 172.21.15.102:0/3935212035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f00bc520 con 0x56399947f640 2024-04-24T23:16:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.607+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3935212035 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56f8028670 con 0x56399947f640 2024-04-24T23:16:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f570b1b0740 10 client.15645 fetch_fsmap learned FSMap version 23 2024-04-24T23:16:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f570b1b0740 1 -- 172.21.15.102:0/3935212035 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56399948e2d0 con 0x56399947f640 2024-04-24T23:16:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3935212035 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56f00bc520 con 0x56399947f640 2024-04-24T23:16:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3935212035 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f56f800f040 con 0x56399947f640 2024-04-24T23:16:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f570b1b0740 10 client.15645 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:16:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f570b1b0740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f570b1b0740 20 client.15645 populate_metadata read hostname 'smithi102' 2024-04-24T23:16:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56399948f790 0x563999491ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f570b1b0740 4 client.15645 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f570b1b0740 1 -- 172.21.15.102:0/3935212035 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639993e21f0 con 0x56399948f790 2024-04-24T23:16:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.608+0000 7f5706243640 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56399948f790 0x563999491ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.609+0000 7f5706243640 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56399948f790 0x563999491ba0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d315a437174d05e8 server_cookie=b6305526658e18b3 in_seq=0 out_seq=0 2024-04-24T23:16:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.609+0000 7f56e9ffb640 10 client.15645 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.609+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3935212035 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639993e21f0 con 0x56399948f790 2024-04-24T23:16:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.609+0000 7f56e9ffb640 10 client.15645 handle_command_reply: tid=0 2024-04-24T23:16:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.609+0000 7f570b1b0740 1 client.15645 shutdown 2024-04-24T23:16:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 -- 172.21.15.102:0/3935212035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 msgr2=0x56399947cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x56399947cfa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f56f8013960 tx=0x7f56f8004ab0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 -- 172.21.15.102:0/3935212035 shutdown_connections 2024-04-24T23:16:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56399948f790 0x563999491ba0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56399947f640 0x56399947cfa0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f56e9ffb640 0 client.15645 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474f70 0x56399947ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3935212035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 0x56399947c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 -- 172.21.15.102:0/3935212035 >> 172.21.15.102:0/3935212035 conn(0x563999470970 msgr2=0x563999476770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 -- 172.21.15.102:0/3935212035 shutdown_connections 2024-04-24T23:16:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 1 -- 172.21.15.102:0/3935212035 wait complete. 2024-04-24T23:16:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.610+0000 7f570b1b0740 20 client.15645 trim_cache size 0 max 16384 2024-04-24T23:16:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f570b1b0740 1 Processor -- start 2024-04-24T23:16:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f570b1b0740 1 -- start start 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 0x56399947ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x56399948af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56399947f640 0x5639994765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f570b1b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f57001ada80 con 0x563999474f70 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f570b1b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57001ab320 con 0x56399947f640 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f570b1b0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f57001a9190 con 0x563999474650 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f5705a42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x56399948af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f5705a42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x56399948af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37948/0 (socket says 172.21.15.102:37948) 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.626+0000 7f5705a42640 1 -- 172.21.15.102:0/3317182581 learned_addr learned my addr 172.21.15.102:0/3317182581 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f5706243640 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 0x56399947ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f5706a44640 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56399947f640 0x5639994765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f5705a42640 1 -- 172.21.15.102:0/3317182581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 msgr2=0x56399947ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f5705a42640 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 0x56399947ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f5705a42640 1 -- 172.21.15.102:0/3317182581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56399947f640 msgr2=0x5639994765c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f5705a42640 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56399947f640 0x5639994765c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f5705a42640 1 -- 172.21.15.102:0/3317182581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563999492220 con 0x563999474f70 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f5706a44640 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56399947f640 0x5639994765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f5705a42640 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x56399948af90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f56f0031440 tx=0x7f56f00dc050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb5c1e684c464cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f56ebfff640 1 -- 172.21.15.102:0/3317182581 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00bb780 con 0x563999474f70 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f56ebfff640 1 -- 172.21.15.102:0/3317182581 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56f00bc520 con 0x563999474f70 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.627+0000 7f56ebfff640 1 -- 172.21.15.102:0/3317182581 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00412c0 con 0x563999474f70 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 -- 172.21.15.102:0/3317182581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 msgr2=0x56399948af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x56399948af90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f56f0031440 tx=0x7f56f00dc050 comp rx=0 tx=0).stop 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 -- 172.21.15.102:0/3317182581 shutdown_connections 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56399947f640 0x5639994765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x56399948af90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3317182581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563999474650 0x56399947ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 -- 172.21.15.102:0/3317182581 >> 172.21.15.102:0/3317182581 conn(0x563999470970 msgr2=0x563999475eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 -- 172.21.15.102:0/3317182581 shutdown_connections 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 -- 172.21.15.102:0/3317182581 wait complete. 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 -- start start 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 Processor -- start 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 0x5639994874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x5639994879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56399947f640 0x563999484620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f57001a97e0 con 0x563999474f70 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57001aac50 con 0x563999474650 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.628+0000 7f570b1b0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f57001a9eb0 con 0x56399947f640 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f570b1b0740 10 client.0 _ll_register_callbacks cb 0x7f56f8009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5705a42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x5639994879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5705a42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x5639994879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37960/0 (socket says 172.21.15.102:37960) 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5705a42640 1 -- 172.21.15.102:0/3402899080 learned_addr learned my addr 172.21.15.102:0/3402899080 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5706a44640 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56399947f640 0x563999484620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5706243640 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 0x5639994874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5705a42640 1 -- 172.21.15.102:0/3402899080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56399947f640 msgr2=0x563999484620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5705a42640 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56399947f640 0x563999484620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5705a42640 1 -- 172.21.15.102:0/3402899080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 msgr2=0x5639994874a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5705a42640 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 0x5639994874a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5705a42640 1 -- 172.21.15.102:0/3402899080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639994707a0 con 0x563999474f70 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f5705a42640 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x5639994879e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f56f00ed650 tx=0x7f56f00bbe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa726517cee8e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f56e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3402899080 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00bb780 con 0x563999474f70 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3402899080 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f56f00d0000 con 0x563999474f70 2024-04-24T23:16:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f570b1b0740 1 -- 172.21.15.102:0/3402899080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f00bc520 con 0x563999474f70 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3402899080 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56f0003e40 con 0x563999474f70 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f570b1b0740 10 client.15651 fetch_fsmap learned FSMap version 23 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.629+0000 7f570b1b0740 1 -- 172.21.15.102:0/3402899080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5639994f66b0 con 0x563999474f70 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3402899080 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56f00bc520 con 0x563999474f70 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3402899080 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7f56f004c060 con 0x563999474f70 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f570b1b0740 10 client.15651 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f570b1b0740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f570b1b0740 20 client.15651 populate_metadata read hostname 'smithi102' 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5639994f69e0 0x5639994f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f570b1b0740 4 client.15651 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f570b1b0740 1 -- 172.21.15.102:0/3402899080 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639993dc540 con 0x5639994f69e0 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f5706243640 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5639994f69e0 0x5639994f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f56e9ffb640 10 client.15651 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.630+0000 7f5706243640 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5639994f69e0 0x5639994f8e00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e13a80480a9a1c6 server_cookie=c33bee009a206665 in_seq=0 out_seq=0 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.638+0000 7f56e9ffb640 1 -- 172.21.15.102:0/3402899080 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639993dc540 con 0x5639994f69e0 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.638+0000 7f56e9ffb640 10 client.15651 handle_command_reply: tid=0 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 client.15651 shutdown 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 -- 172.21.15.102:0/3402899080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 msgr2=0x5639994879e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x5639994879e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f56f00ed650 tx=0x7f56f00bbe20 comp rx=0 tx=0).stop 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 -- 172.21.15.102:0/3402899080 shutdown_connections 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56399947f640 0x563999484620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5639994f69e0 0x5639994f8e00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563999474f70 0x5639994879e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 --2- 172.21.15.102:0/3402899080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563999474650 0x5639994874a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f56e9ffb640 0 client.15651 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 -- 172.21.15.102:0/3402899080 >> 172.21.15.102:0/3402899080 conn(0x563999470970 msgr2=0x56399946b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 -- 172.21.15.102:0/3402899080 shutdown_connections 2024-04-24T23:16:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 1 -- 172.21.15.102:0/3402899080 wait complete. 2024-04-24T23:16:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.639+0000 7f570b1b0740 20 client.15651 trim_cache size 0 max 16384 2024-04-24T23:16:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:16:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:16:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:16:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:16:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 -- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f56ec0a2da0 msgr2=0x7f56ec0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f56ec0a2da0 0x7f56ec0a5260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56f80098b0 tx=0x7f56f8004a40 comp rx=0 tx=0).stop 2024-04-24T23:16:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 -- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700101030 msgr2=0x7f57001984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700101030 0x7f57001984f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f56f0005ab0 tx=0x7f56f0003cd0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 -- 172.21.15.102:0/1085057336 shutdown_connections 2024-04-24T23:16:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5700106630 0x7f570019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f56ec0a2da0 0x7f56ec0a5260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5700101030 0x7f57001984f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 --2- 172.21.15.102:0/1085057336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5700100670 0x7f5700197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 -- 172.21.15.102:0/1085057336 >> 172.21.15.102:0/1085057336 conn(0x7f57000777c0 msgr2=0x7f5700069230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 -- 172.21.15.102:0/1085057336 shutdown_connections 2024-04-24T23:16:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:08.640+0000 7f57084a7640 1 -- 172.21.15.102:0/1085057336 wait complete. 2024-04-24T23:16:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:08 smithi183 ceph-mon[33137]: pgmap v96: 97 pgs: 97 active+clean; 2.7 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 MiB/s wr, 39 op/s 2024-04-24T23:16:09.073 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:08 smithi102 ceph-mon[29216]: pgmap v96: 97 pgs: 97 active+clean; 2.7 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 MiB/s wr, 39 op/s 2024-04-24T23:16:09.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:08 smithi161 ceph-mon[32709]: pgmap v96: 97 pgs: 97 active+clean; 2.7 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 MiB/s wr, 39 op/s 2024-04-24T23:16:10.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:09 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:10.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:09 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:10.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:09 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:11.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:10 smithi183 ceph-mon[33137]: pgmap v97: 97 pgs: 97 active+clean; 2.7 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-04-24T23:16:11.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:10 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:11.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:10 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:11.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:10 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:16:11.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:10 smithi102 ceph-mon[29216]: pgmap v97: 97 pgs: 97 active+clean; 2.7 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-04-24T23:16:11.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:10 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:11.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:10 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:11.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:10 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:16:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:10 smithi161 ceph-mon[32709]: pgmap v97: 97 pgs: 97 active+clean; 2.7 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-04-24T23:16:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:10 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:10 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:10 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:16:13.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:12 smithi183 ceph-mon[33137]: pgmap v98: 97 pgs: 97 active+clean; 2.7 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-04-24T23:16:13.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:12 smithi102 ceph-mon[29216]: pgmap v98: 97 pgs: 97 active+clean; 2.7 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-04-24T23:16:13.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:12 smithi161 ceph-mon[32709]: pgmap v98: 97 pgs: 97 active+clean; 2.7 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-04-24T23:16:15.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:14 smithi183 ceph-mon[33137]: pgmap v99: 97 pgs: 97 active+clean; 2.9 GiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-04-24T23:16:15.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:14 smithi102 ceph-mon[29216]: pgmap v99: 97 pgs: 97 active+clean; 2.9 GiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-04-24T23:16:15.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:14 smithi161 ceph-mon[32709]: pgmap v99: 97 pgs: 97 active+clean; 2.9 GiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-04-24T23:16:17.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:16 smithi183 ceph-mon[33137]: pgmap v100: 97 pgs: 97 active+clean; 2.9 GiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:17.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:16 smithi102 ceph-mon[29216]: pgmap v100: 97 pgs: 97 active+clean; 2.9 GiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:17.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:16 smithi161 ceph-mon[32709]: pgmap v100: 97 pgs: 97 active+clean; 2.9 GiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:19.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:18 smithi183 ceph-mon[33137]: pgmap v101: 97 pgs: 97 active+clean; 3.0 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:19.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:18 smithi102 ceph-mon[29216]: pgmap v101: 97 pgs: 97 active+clean; 3.0 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:19.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:18 smithi161 ceph-mon[32709]: pgmap v101: 97 pgs: 97 active+clean; 3.0 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:21.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:20 smithi183 ceph-mon[33137]: pgmap v102: 97 pgs: 97 active+clean; 3.0 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:21.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:20 smithi102 ceph-mon[29216]: pgmap v102: 97 pgs: 97 active+clean; 3.0 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:21.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:20 smithi161 ceph-mon[32709]: pgmap v102: 97 pgs: 97 active+clean; 3.0 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.733+0000 7fd6de488640 1 -- 172.21.15.102:0/3352030133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 msgr2=0x7fd6d810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.733+0000 7fd6de488640 1 --2- 172.21.15.102:0/3352030133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 0x7fd6d810d4f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009cd0 tx=0x7fd6d002a8e0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.733+0000 7fd6de488640 1 -- 172.21.15.102:0/3352030133 shutdown_connections 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.733+0000 7fd6de488640 1 --2- 172.21.15.102:0/3352030133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 0x7fd6d810d4f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.733+0000 7fd6de488640 1 --2- 172.21.15.102:0/3352030133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d8100670 0x7fd6d8100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.733+0000 7fd6de488640 1 --2- 172.21.15.102:0/3352030133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d8106630 0x7fd6d8106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.733+0000 7fd6de488640 1 -- 172.21.15.102:0/3352030133 >> 172.21.15.102:0/3352030133 conn(0x7fd6d80777c0 msgr2=0x7fd6d8077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.733+0000 7fd6de488640 1 -- 172.21.15.102:0/3352030133 shutdown_connections 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.733+0000 7fd6de488640 1 -- 172.21.15.102:0/3352030133 wait complete. 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6de488640 1 Processor -- start 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6de488640 1 -- start start 2024-04-24T23:16:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6de488640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d8100670 0x7fd6d8196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6de488640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 0x7fd6d8196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6de488640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d8106630 0x7fd6d8197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6de488640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d8110900 con 0x7fd6d8101030 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6de488640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d8110780 con 0x7fd6d8106630 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6de488640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d8110a80 con 0x7fd6d8100670 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6d77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 0x7fd6d8196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6d77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 0x7fd6d8196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49410/0 (socket says 172.21.15.102:49410) 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6d77fe640 1 -- 172.21.15.102:0/3405181537 learned_addr learned my addr 172.21.15.102:0/3405181537 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d8100670 0x7fd6d8196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.734+0000 7fd6dca25640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d8106630 0x7fd6d8197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6d77fe640 1 -- 172.21.15.102:0/3405181537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d8100670 msgr2=0x7fd6d8196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d8100670 0x7fd6d8196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6d77fe640 1 -- 172.21.15.102:0/3405181537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d8106630 msgr2=0x7fd6d8197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d8106630 0x7fd6d8197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6d77fe640 1 -- 172.21.15.102:0/3405181537 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d81a6320 con 0x7fd6d8101030 2024-04-24T23:16:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 0x7fd6d8196ff0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00d950 tx=0x7fd6cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6d57fa640 1 -- 172.21.15.102:0/3405181537 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc017070 con 0x7fd6d8101030 2024-04-24T23:16:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6de488640 1 -- 172.21.15.102:0/3405181537 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d81a6610 con 0x7fd6d8101030 2024-04-24T23:16:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6d57fa640 1 -- 172.21.15.102:0/3405181537 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6cc009c30 con 0x7fd6d8101030 2024-04-24T23:16:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6d57fa640 1 -- 172.21.15.102:0/3405181537 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00f620 con 0x7fd6d8101030 2024-04-24T23:16:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.735+0000 7fd6de488640 1 -- 172.21.15.102:0/3405181537 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d81a6b80 con 0x7fd6d8101030 2024-04-24T23:16:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.737+0000 7fd6d57fa640 1 -- 172.21.15.102:0/3405181537 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd6cc0109a0 con 0x7fd6d8101030 2024-04-24T23:16:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6e1190740 1 Processor -- start 2024-04-24T23:16:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6e1190740 1 -- start start 2024-04-24T23:16:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d57fa640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6c00a2da0 0x7fd6c00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 0x55ff514da620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6dca25640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6c00a2da0 0x7fd6c00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6e1190740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ff5143ecb0 con 0x55ff514dab60 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6e1190740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ff513e75b0 con 0x55ff514e5230 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6e1190740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ff5143ac70 con 0x55ff514da240 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d57fa640 1 -- 172.21.15.102:0/3405181537 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fd6cc0ceff0 con 0x7fd6d8101030 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49424/0 (socket says 172.21.15.102:49424) 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d77fe640 1 -- 172.21.15.102:0/501361282 learned_addr learned my addr 172.21.15.102:0/501361282 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 0x55ff514da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6dca25640 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d77fe640 1 -- 172.21.15.102:0/501361282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 msgr2=0x55ff514da620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 0x55ff514da620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d77fe640 1 -- 172.21.15.102:0/501361282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 msgr2=0x55ff514e7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6d77fe640 1 -- 172.21.15.102:0/501361282 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff51153210 con 0x55ff514dab60 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.738+0000 7fd6dca25640 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.739+0000 7fd6dca25640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6c00a2da0 0x7fd6c00a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0004af0 tx=0x7fd6d0005f10 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.739+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e4cf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc097f80 tx=0x7fd6cc009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e690f768d8584f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.739+0000 7fd6be7fc640 1 -- 172.21.15.102:0/501361282 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc020040 con 0x55ff514dab60 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.739+0000 7fd6be7fc640 1 -- 172.21.15.102:0/501361282 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6cc09f8b0 con 0x55ff514dab60 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.739+0000 7fd6be7fc640 1 -- 172.21.15.102:0/501361282 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc084d10 con 0x55ff514dab60 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.739+0000 7fd6e1190740 1 -- 172.21.15.102:0/501361282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 msgr2=0x55ff514e4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.739+0000 7fd6e1190740 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e4cf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc097f80 tx=0x7fd6cc009de0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 -- 172.21.15.102:0/501361282 shutdown_connections 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e4cf0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 --2- 172.21.15.102:0/501361282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 0x55ff514da620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 -- 172.21.15.102:0/501361282 >> 172.21.15.102:0/501361282 conn(0x55ff514d6560 msgr2=0x55ff514d6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 -- 172.21.15.102:0/501361282 shutdown_connections 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 -- 172.21.15.102:0/501361282 wait complete. 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 -- start start 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 Processor -- start 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514da240 0x55ff514e2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514dab60 0x55ff514e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ff514e7fc0 con 0x55ff514da240 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ff513fab50 con 0x55ff514e5230 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ff514e8140 con 0x55ff514dab60 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6e1190740 10 client.0 _ll_register_callbacks cb 0x55ff51435ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514da240 0x55ff514e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.740+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514da240 0x55ff514e2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49430/0 (socket says 172.21.15.102:49430) 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6d7fff640 1 -- 172.21.15.102:0/1096321049 learned_addr learned my addr 172.21.15.102:0/1096321049 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514dab60 0x55ff514e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6dca25640 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6d7fff640 1 -- 172.21.15.102:0/1096321049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514dab60 msgr2=0x55ff514e26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514dab60 0x55ff514e26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6d7fff640 1 -- 172.21.15.102:0/1096321049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 msgr2=0x55ff514e2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6d7fff640 1 -- 172.21.15.102:0/1096321049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff5121f620 con 0x55ff514da240 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6dca25640 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514da240 0x55ff514e2180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800bf80 tx=0x7fd6c80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea33df6e3e7bcaa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1096321049 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8004020 con 0x55ff514da240 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1096321049 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fd6c80041c0 con 0x55ff514da240 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6e1190740 1 -- 172.21.15.102:0/1096321049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6cc09f8b0 con 0x55ff514da240 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1096321049 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd6c80043c0 con 0x55ff514da240 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6e1190740 10 client.15663 fetch_fsmap learned FSMap version 23 2024-04-24T23:16:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.741+0000 7fd6e1190740 1 -- 172.21.15.102:0/1096321049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ff514f40f0 con 0x55ff514da240 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1096321049 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6cc09f8b0 con 0x55ff514da240 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1096321049 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7fd6c8003990 con 0x55ff514da240 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6e1190740 10 client.15663 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6e1190740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6e1190740 20 client.15663 populate_metadata read hostname 'smithi102' 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ff514f5670 0x55ff514f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6e1190740 4 client.15663 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6e1190740 1 -- 172.21.15.102:0/1096321049 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff5144c1f0 con 0x55ff514f5670 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ff514f5670 0x55ff514f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ff514f5670 0x55ff514f7a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7e1ba0309987e9 server_cookie=b2c0e804627fe1c in_seq=0 out_seq=0 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.742+0000 7fd6b77fe640 10 client.15663 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.743+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1096321049 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff5144c1f0 con 0x55ff514f5670 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.743+0000 7fd6b77fe640 10 client.15663 handle_command_reply: tid=0 2024-04-24T23:16:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.743+0000 7fd6e1190740 1 client.15663 shutdown 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.743+0000 7fd6e1190740 1 -- 172.21.15.102:0/1096321049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514da240 msgr2=0x55ff514e2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.743+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514da240 0x55ff514e2180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800bf80 tx=0x7fd6c80118d0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.743+0000 7fd6e1190740 1 -- 172.21.15.102:0/1096321049 shutdown_connections 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.743+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514e5230 0x55ff514e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.743+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514dab60 0x55ff514e26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.744+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ff514f5670 0x55ff514f7a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.744+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1096321049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514da240 0x55ff514e2180 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.744+0000 7fd6b77fe640 0 client.15663 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.744+0000 7fd6e1190740 1 -- 172.21.15.102:0/1096321049 >> 172.21.15.102:0/1096321049 conn(0x55ff514d6560 msgr2=0x55ff514dc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.744+0000 7fd6e1190740 1 -- 172.21.15.102:0/1096321049 shutdown_connections 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.744+0000 7fd6e1190740 1 -- 172.21.15.102:0/1096321049 wait complete. 2024-04-24T23:16:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.744+0000 7fd6e1190740 20 client.15663 trim_cache size 0 max 16384 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.758+0000 7fd6e1190740 1 Processor -- start 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.758+0000 7fd6e1190740 1 -- start start 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 0x55ff514f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514dab60 0x55ff514dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514e5230 0x55ff514d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6e1190740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d81b1f10 con 0x55ff514e5230 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6e1190740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d81af7b0 con 0x55ff514dab60 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6e1190740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d81ad620 con 0x55ff514da240 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6dca25640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514e5230 0x55ff514d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6dca25640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514e5230 0x55ff514d5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49444/0 (socket says 172.21.15.102:49444) 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6dca25640 1 -- 172.21.15.102:0/4134356736 learned_addr learned my addr 172.21.15.102:0/4134356736 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514dab60 0x55ff514dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 0x55ff514f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6dca25640 1 -- 172.21.15.102:0/4134356736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 msgr2=0x55ff514f0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6dca25640 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 0x55ff514f0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6dca25640 1 -- 172.21.15.102:0/4134356736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514dab60 msgr2=0x55ff514dcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6dca25640 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514dab60 0x55ff514dcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6dca25640 1 -- 172.21.15.102:0/4134356736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff514e06d0 con 0x55ff514e5230 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514dab60 0x55ff514dcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 0x55ff514f0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6dca25640 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514e5230 0x55ff514d5440 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0035000 tx=0x7fd6d0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43dd2c41cb959ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.759+0000 7fd6bdffb640 1 -- 172.21.15.102:0/4134356736 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0004240 con 0x55ff514e5230 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6bdffb640 1 -- 172.21.15.102:0/4134356736 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d0036070 con 0x55ff514e5230 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6bdffb640 1 -- 172.21.15.102:0/4134356736 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0030c90 con 0x55ff514e5230 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6e1190740 1 -- 172.21.15.102:0/4134356736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514e5230 msgr2=0x55ff514d5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6e1190740 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514e5230 0x55ff514d5440 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0035000 tx=0x7fd6d0040000 comp rx=0 tx=0).stop 2024-04-24T23:16:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6e1190740 1 -- 172.21.15.102:0/4134356736 shutdown_connections 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6e1190740 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514e5230 0x55ff514d5440 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6e1190740 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514dab60 0x55ff514dcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6e1190740 1 --2- 172.21.15.102:0/4134356736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514da240 0x55ff514f0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6e1190740 1 -- 172.21.15.102:0/4134356736 >> 172.21.15.102:0/4134356736 conn(0x55ff514d6560 msgr2=0x55ff514d4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6e1190740 1 -- 172.21.15.102:0/4134356736 shutdown_connections 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.760+0000 7fd6e1190740 1 -- 172.21.15.102:0/4134356736 wait complete. 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6e1190740 1 -- start start 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6e1190740 1 Processor -- start 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514da240 0x55ff514e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6e1190740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514e5230 0x55ff514e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6e1190740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d81adc70 con 0x55ff514dab60 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6e1190740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d81af0e0 con 0x55ff514da240 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6e1190740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d81ae340 con 0x55ff514e5230 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6e1190740 10 client.0 _ll_register_callbacks cb 0x7fd6c800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6d77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6d77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49454/0 (socket says 172.21.15.102:49454) 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.761+0000 7fd6d77fe640 1 -- 172.21.15.102:0/1137400445 learned_addr learned my addr 172.21.15.102:0/1137400445 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6dca25640 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514e5230 0x55ff514e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514da240 0x55ff514e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6d77fe640 1 -- 172.21.15.102:0/1137400445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514e5230 msgr2=0x55ff514e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514e5230 0x55ff514e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6d77fe640 1 -- 172.21.15.102:0/1137400445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514da240 msgr2=0x55ff514e8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514da240 0x55ff514e8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6d77fe640 1 -- 172.21.15.102:0/1137400445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff5155ac10 con 0x55ff514dab60 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514da240 0x55ff514e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6d77fe640 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e9350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc097a10 tx=0x7fd6cc097980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb1c25377d616e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1137400445 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc020040 con 0x55ff514dab60 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1137400445 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fd6cc09aa00 con 0x55ff514dab60 2024-04-24T23:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6e1190740 1 -- 172.21.15.102:0/1137400445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d0036070 con 0x55ff514dab60 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1137400445 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd6cc09f8b0 con 0x55ff514dab60 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6e1190740 10 client.15669 fetch_fsmap learned FSMap version 23 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.762+0000 7fd6e1190740 1 -- 172.21.15.102:0/1137400445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ff5155bf20 con 0x55ff514dab60 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1137400445 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d0036070 con 0x55ff514dab60 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1137400445 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7fd6cc010460 con 0x55ff514dab60 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6e1190740 10 client.15669 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6e1190740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6e1190740 20 client.15669 populate_metadata read hostname 'smithi102' 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ff5155c2b0 0x55ff5155e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6e1190740 4 client.15669 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6e1190740 1 -- 172.21.15.102:0/1137400445 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff51446540 con 0x55ff5155c2b0 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ff5155c2b0 0x55ff5155e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6b77fe640 10 client.15669 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.763+0000 7fd6d7fff640 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ff5155c2b0 0x55ff5155e6a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf36969234fe082 server_cookie=e86333d870388697 in_seq=0 out_seq=0 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6b77fe640 1 -- 172.21.15.102:0/1137400445 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff51446540 con 0x55ff5155c2b0 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6b77fe640 10 client.15669 handle_command_reply: tid=0 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 client.15669 shutdown 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 -- 172.21.15.102:0/1137400445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 msgr2=0x55ff514e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e9350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc097a10 tx=0x7fd6cc097980 comp rx=0 tx=0).stop 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 -- 172.21.15.102:0/1137400445 shutdown_connections 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff514e5230 0x55ff514e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ff5155c2b0 0x55ff5155e6a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff514dab60 0x55ff514e9350 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6b77fe640 0 client.15669 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 --2- 172.21.15.102:0/1137400445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ff514da240 0x55ff514e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 -- 172.21.15.102:0/1137400445 >> 172.21.15.102:0/1137400445 conn(0x55ff514d6560 msgr2=0x55ff514d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 -- 172.21.15.102:0/1137400445 shutdown_connections 2024-04-24T23:16:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.771+0000 7fd6e1190740 1 -- 172.21.15.102:0/1137400445 wait complete. 2024-04-24T23:16:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6e1190740 20 client.15669 trim_cache size 0 max 16384 2024-04-24T23:16:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:16:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:16:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:16:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:16:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 -- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6c00a2da0 msgr2=0x7fd6c00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6c00a2da0 0x7fd6c00a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0004af0 tx=0x7fd6d0005f10 comp rx=0 tx=0).stop 2024-04-24T23:16:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 -- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 msgr2=0x7fd6d8196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 0x7fd6d8196ff0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00d950 tx=0x7fd6cc00de10 comp rx=0 tx=0).stop 2024-04-24T23:16:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 -- 172.21.15.102:0/3405181537 shutdown_connections 2024-04-24T23:16:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d8106630 0x7fd6d8197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6c00a2da0 0x7fd6c00a5260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6d8101030 0x7fd6d8196ff0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 --2- 172.21.15.102:0/3405181537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6d8100670 0x7fd6d8196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 -- 172.21.15.102:0/3405181537 >> 172.21.15.102:0/3405181537 conn(0x7fd6d80777c0 msgr2=0x7fd6d81030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 -- 172.21.15.102:0/3405181537 shutdown_connections 2024-04-24T23:16:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:21.772+0000 7fd6de488640 1 -- 172.21.15.102:0/3405181537 wait complete. 2024-04-24T23:16:23.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:22 smithi183 ceph-mon[33137]: pgmap v103: 97 pgs: 97 active+clean; 3.0 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:23.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:22 smithi102 ceph-mon[29216]: pgmap v103: 97 pgs: 97 active+clean; 3.0 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:23.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:22 smithi161 ceph-mon[32709]: pgmap v103: 97 pgs: 97 active+clean; 3.0 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:25.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:24 smithi183 ceph-mon[33137]: pgmap v104: 97 pgs: 97 active+clean; 3.2 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 26 op/s 2024-04-24T23:16:25.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:24 smithi102 ceph-mon[29216]: pgmap v104: 97 pgs: 97 active+clean; 3.2 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 26 op/s 2024-04-24T23:16:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:24 smithi161 ceph-mon[32709]: pgmap v104: 97 pgs: 97 active+clean; 3.2 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 26 op/s 2024-04-24T23:16:27.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:26 smithi183 ceph-mon[33137]: pgmap v105: 97 pgs: 97 active+clean; 3.2 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:16:27.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:26 smithi102 ceph-mon[29216]: pgmap v105: 97 pgs: 97 active+clean; 3.2 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:16:27.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:26 smithi161 ceph-mon[32709]: pgmap v105: 97 pgs: 97 active+clean; 3.2 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:16:29.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:28 smithi183 ceph-mon[33137]: pgmap v106: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:16:29.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:28 smithi102 ceph-mon[29216]: pgmap v106: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:16:29.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:28 smithi161 ceph-mon[32709]: pgmap v106: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:16:31.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:30 smithi183 ceph-mon[33137]: pgmap v107: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:31.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:30 smithi102 ceph-mon[29216]: pgmap v107: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:31.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:30 smithi161 ceph-mon[32709]: pgmap v107: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:32 smithi183 ceph-mon[33137]: pgmap v108: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:33.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:32 smithi102 ceph-mon[29216]: pgmap v108: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:33.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:32 smithi161 ceph-mon[32709]: pgmap v108: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 -- 172.21.15.102:0/2752302400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0101030 msgr2=0x7fc2c010d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/2752302400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0101030 0x7fc2c010d4f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009cd0 tx=0x7fc2b802a8e0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 -- 172.21.15.102:0/2752302400 shutdown_connections 2024-04-24T23:16:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/2752302400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0101030 0x7fc2c010d4f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/2752302400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2c0100670 0x7fc2c0100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/2752302400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0106630 0x7fc2c0106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 -- 172.21.15.102:0/2752302400 >> 172.21.15.102:0/2752302400 conn(0x7fc2c00777c0 msgr2=0x7fc2c0077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 -- 172.21.15.102:0/2752302400 shutdown_connections 2024-04-24T23:16:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 -- 172.21.15.102:0/2752302400 wait complete. 2024-04-24T23:16:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.868+0000 7fc2c5d42640 1 Processor -- start 2024-04-24T23:16:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2c5d42640 1 -- start start 2024-04-24T23:16:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2c5d42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0100670 0x7fc2c0197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2c5d42640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2c0101030 0x7fc2c0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2c5d42640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0106630 0x7fc2c019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2c5d42640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01108b0 con 0x7fc2c0100670 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2c5d42640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0110730 con 0x7fc2c0101030 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2c5d42640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0110a30 con 0x7fc2c0106630 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2bf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0100670 0x7fc2c0197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2bf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0100670 0x7fc2c0197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33950/0 (socket says 172.21.15.102:33950) 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2bf7fe640 1 -- 172.21.15.102:0/850582602 learned_addr learned my addr 172.21.15.102:0/850582602 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2beffd640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2c0101030 0x7fc2c0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2bffff640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0106630 0x7fc2c019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2bf7fe640 1 -- 172.21.15.102:0/850582602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0106630 msgr2=0x7fc2c019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0106630 0x7fc2c019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2bf7fe640 1 -- 172.21.15.102:0/850582602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2c0101030 msgr2=0x7fc2c0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2c0101030 0x7fc2c0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.869+0000 7fc2bf7fe640 1 -- 172.21.15.102:0/850582602 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c019ce40 con 0x7fc2c0100670 2024-04-24T23:16:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.870+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0100670 0x7fc2c0197e50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000ea60 tx=0x7fc2b000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.870+0000 7fc2bcff9640 1 -- 172.21.15.102:0/850582602 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000cde0 con 0x7fc2c0100670 2024-04-24T23:16:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.870+0000 7fc2bcff9640 1 -- 172.21.15.102:0/850582602 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2b00035f0 con 0x7fc2c0100670 2024-04-24T23:16:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.870+0000 7fc2c5d42640 1 -- 172.21.15.102:0/850582602 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c019d130 con 0x7fc2c0100670 2024-04-24T23:16:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.870+0000 7fc2bcff9640 1 -- 172.21.15.102:0/850582602 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0010690 con 0x7fc2c0100670 2024-04-24T23:16:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.870+0000 7fc2c5d42640 1 -- 172.21.15.102:0/850582602 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c01a49c0 con 0x7fc2c0100670 2024-04-24T23:16:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.872+0000 7fc2bcff9640 1 -- 172.21.15.102:0/850582602 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fc2b00037a0 con 0x7fc2c0100670 2024-04-24T23:16:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2c8a52740 1 Processor -- start 2024-04-24T23:16:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2c8a52740 1 -- start start 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966baf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bbc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555966b13cb0 con 0x555966bba230 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555966abc5b0 con 0x555966baf240 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555966b0fc70 con 0x555966bafb60 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2bcff9640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc2a80a2da0 0x7fc2a80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2beffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bbc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2beffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bbc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33962/0 (socket says 172.21.15.102:33962) 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2beffd640 1 -- 172.21.15.102:0/548041658 learned_addr learned my addr 172.21.15.102:0/548041658 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2bffff640 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966baf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.873+0000 7fc2bffff640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc2a80a2da0 0x7fc2a80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2bcff9640 1 -- 172.21.15.102:0/850582602 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fc2b00cf6e0 con 0x7fc2c0100670 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2beffd640 1 -- 172.21.15.102:0/548041658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 msgr2=0x555966bb9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2beffd640 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2beffd640 1 -- 172.21.15.102:0/548041658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 msgr2=0x555966baf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2beffd640 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966baf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2beffd640 1 -- 172.21.15.102:0/548041658 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555966828210 con 0x555966bba230 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2bffff640 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966baf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2beffd640 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bbc620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400bcb0 tx=0x7fc2b40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69103f09aef5c942 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2bffff640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc2a80a2da0 0x7fc2a80a5260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8033040 tx=0x7fc2b802a8e0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2a5ffb640 1 -- 172.21.15.102:0/548041658 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4004030 con 0x555966bba230 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2a5ffb640 1 -- 172.21.15.102:0/548041658 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2b40041d0 con 0x555966bba230 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.874+0000 7fc2a5ffb640 1 -- 172.21.15.102:0/548041658 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4011680 con 0x555966bba230 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 -- 172.21.15.102:0/548041658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 msgr2=0x555966bbc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bbc620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400bcb0 tx=0x7fc2b40107a0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 -- 172.21.15.102:0/548041658 shutdown_connections 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bbc620 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/548041658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966baf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 -- 172.21.15.102:0/548041658 >> 172.21.15.102:0/548041658 conn(0x555966bab560 msgr2=0x555966bab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 -- 172.21.15.102:0/548041658 shutdown_connections 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 -- 172.21.15.102:0/548041658 wait complete. 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 -- start start 2024-04-24T23:16:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.875+0000 7fc2c8a52740 1 Processor -- start 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966bb7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bb7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555966bbd020 con 0x555966bba230 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555966acfb50 con 0x555966baf240 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555966bbd1a0 con 0x555966bafb60 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2c8a52740 10 client.0 _ll_register_callbacks cb 0x555966b0aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bb7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bb7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33972/0 (socket says 172.21.15.102:33972) 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bffff640 1 -- 172.21.15.102:0/28640964 learned_addr learned my addr 172.21.15.102:0/28640964 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966bb7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2beffd640 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bffff640 1 -- 172.21.15.102:0/28640964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 msgr2=0x555966bb7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bffff640 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bffff640 1 -- 172.21.15.102:0/28640964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 msgr2=0x555966bb7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bffff640 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966bb7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bffff640 1 -- 172.21.15.102:0/28640964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559668f4620 con 0x555966bba230 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966bb7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc2bffff640 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bb7bb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8035000 tx=0x7fc2b80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5316162d51bd0d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.876+0000 7fc29effd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc29effd640 1 -- 172.21.15.102:0/28640964 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b804f070 con 0x555966bba230 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc29effd640 1 -- 172.21.15.102:0/28640964 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fc2b8004310 con 0x555966bba230 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2c8a52740 1 -- 172.21.15.102:0/28640964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b40041d0 con 0x555966bba230 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc29effd640 1 -- 172.21.15.102:0/28640964 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2b8004510 con 0x555966bba230 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2c8a52740 10 client.15681 fetch_fsmap learned FSMap version 23 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2c8a52740 1 -- 172.21.15.102:0/28640964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555966bcd5a0 con 0x555966bba230 2024-04-24T23:16:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc29effd640 1 -- 172.21.15.102:0/28640964 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2b40041d0 con 0x555966bba230 2024-04-24T23:16:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc29effd640 1 -- 172.21.15.102:0/28640964 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7fc2b8030070 con 0x555966bba230 2024-04-24T23:16:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2c8a52740 10 client.15681 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:16:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2c8a52740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2c8a52740 20 client.15681 populate_metadata read hostname 'smithi102' 2024-04-24T23:16:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555966bceb20 0x555966bd0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2c8a52740 4 client.15681 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2c8a52740 1 -- 172.21.15.102:0/28640964 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555966b211f0 con 0x555966bceb20 2024-04-24T23:16:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.877+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555966bceb20 0x555966bd0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.878+0000 7fc29effd640 10 client.15681 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.878+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555966bceb20 0x555966bd0f30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3bf2ae608747d5 server_cookie=6639fdb6399929b6 in_seq=0 out_seq=0 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc29effd640 1 -- 172.21.15.102:0/28640964 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555966b211f0 con 0x555966bceb20 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc29effd640 10 client.15681 handle_command_reply: tid=0 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 client.15681 shutdown 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 -- 172.21.15.102:0/28640964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 msgr2=0x555966bb7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bb7bb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8035000 tx=0x7fc2b80095c0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 -- 172.21.15.102:0/28640964 shutdown_connections 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555966bceb20 0x555966bd0f30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bba230 0x555966bb7bb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc29effd640 0 client.15681 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/28640964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966bb7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 -- 172.21.15.102:0/28640964 >> 172.21.15.102:0/28640964 conn(0x555966bab560 msgr2=0x555966bb1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 -- 172.21.15.102:0/28640964 shutdown_connections 2024-04-24T23:16:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 1 -- 172.21.15.102:0/28640964 wait complete. 2024-04-24T23:16:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.879+0000 7fc2c8a52740 20 client.15681 trim_cache size 0 max 16384 2024-04-24T23:16:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.898+0000 7fc2c8a52740 1 Processor -- start 2024-04-24T23:16:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.898+0000 7fc2c8a52740 1 -- start start 2024-04-24T23:16:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.898+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966baf240 0x555966bb9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.898+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.898+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966bba230 0x555966baa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.898+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01ada60 con 0x555966baf240 2024-04-24T23:16:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.898+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01ab300 con 0x555966bba230 2024-04-24T23:16:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.898+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01a9190 con 0x555966bafb60 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966baf240 0x555966bb9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966baf240 0x555966bb9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33974/0 (socket says 172.21.15.102:33974) 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2beffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bffff640 1 --2- 172.21.15.102:0/2834647723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966bba230 0x555966baa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bf7fe640 1 -- 172.21.15.102:0/2834647723 learned_addr learned my addr 172.21.15.102:0/2834647723 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bffff640 1 -- 172.21.15.102:0/2834647723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 msgr2=0x555966bb6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bffff640 1 --2- 172.21.15.102:0/2834647723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bffff640 1 -- 172.21.15.102:0/2834647723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966baf240 msgr2=0x555966bb9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bffff640 1 --2- 172.21.15.102:0/2834647723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966baf240 0x555966bb9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bffff640 1 -- 172.21.15.102:0/2834647723 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555966bba060 con 0x555966bba230 2024-04-24T23:16:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2bffff640 1 --2- 172.21.15.102:0/2834647723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966bba230 0x555966baa440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8003f70 tx=0x7fc2b8031dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1741002cfbe2b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2a57fa640 1 -- 172.21.15.102:0/2834647723 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b804f070 con 0x555966bba230 2024-04-24T23:16:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2a57fa640 1 -- 172.21.15.102:0/2834647723 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2b8036040 con 0x555966bba230 2024-04-24T23:16:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.899+0000 7fc2a57fa640 1 -- 172.21.15.102:0/2834647723 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80046b0 con 0x555966bba230 2024-04-24T23:16:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 -- 172.21.15.102:0/2834647723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966bba230 msgr2=0x555966baa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/2834647723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966bba230 0x555966baa440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8003f70 tx=0x7fc2b8031dd0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 -- 172.21.15.102:0/2834647723 shutdown_connections 2024-04-24T23:16:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/2834647723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966bba230 0x555966baa440 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/2834647723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bafb60 0x555966bb6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/2834647723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966baf240 0x555966bb9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 -- 172.21.15.102:0/2834647723 >> 172.21.15.102:0/2834647723 conn(0x555966bab560 msgr2=0x555966ba9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 -- 172.21.15.102:0/2834647723 shutdown_connections 2024-04-24T23:16:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 -- 172.21.15.102:0/2834647723 wait complete. 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.900+0000 7fc2c8a52740 1 -- start start 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2c8a52740 1 Processor -- start 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966bc6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bafb60 0x555966bb39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2c8a52740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bba230 0x555966bb3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01a97e0 con 0x555966bafb60 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01aac30 con 0x555966baf240 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2c8a52740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01a9e90 con 0x555966bba230 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2c8a52740 10 client.0 _ll_register_callbacks cb 0x7fc2b8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2beffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bafb60 0x555966bb39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2beffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bafb60 0x555966bb39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33988/0 (socket says 172.21.15.102:33988) 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2beffd640 1 -- 172.21.15.102:0/3726257279 learned_addr learned my addr 172.21.15.102:0/3726257279 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2bffff640 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bba230 0x555966bb3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966bc6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2beffd640 1 -- 172.21.15.102:0/3726257279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bba230 msgr2=0x555966bb3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2beffd640 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bba230 0x555966bb3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2beffd640 1 -- 172.21.15.102:0/3726257279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 msgr2=0x555966bc6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2beffd640 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966bc6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.901+0000 7fc2beffd640 1 -- 172.21.15.102:0/3726257279 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555966bb1ec0 con 0x555966bafb60 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2beffd640 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bafb60 0x555966bb39a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400bcb0 tx=0x7fc2b400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22963a225f2cda9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc29effd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:16:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc29effd640 1 -- 172.21.15.102:0/3726257279 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b402b060 con 0x555966bafb60 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc29effd640 1 -- 172.21.15.102:0/3726257279 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fc2b4005040 con 0x555966bafb60 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2c8a52740 1 -- 172.21.15.102:0/3726257279 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b8036040 con 0x555966bafb60 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc29effd640 1 -- 172.21.15.102:0/3726257279 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2b4003ac0 con 0x555966bafb60 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2c8a52740 10 client.15687 fetch_fsmap learned FSMap version 23 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2c8a52740 1 -- 172.21.15.102:0/3726257279 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555966c34c00 con 0x555966bafb60 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc29effd640 1 -- 172.21.15.102:0/3726257279 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2b8036040 con 0x555966bafb60 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc29effd640 1 -- 172.21.15.102:0/3726257279 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) ==== 8350+0+0 (secure 0 0 0) 0x7fc2b4011bd0 con 0x555966bafb60 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2c8a52740 10 client.15687 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2c8a52740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2c8a52740 20 client.15687 populate_metadata read hostname 'smithi102' 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555966c350a0 0x555966c374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2c8a52740 4 client.15687 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:16:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.902+0000 7fc2c8a52740 1 -- 172.21.15.102:0/3726257279 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555966b1b540 con 0x555966c350a0 2024-04-24T23:16:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.903+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555966c350a0 0x555966c374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.903+0000 7fc29effd640 10 client.15687 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.903+0000 7fc2bf7fe640 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555966c350a0 0x555966c374c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf08aa82a7fc04d server_cookie=79c1b69fd2746f63 in_seq=0 out_seq=0 2024-04-24T23:16:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc29effd640 1 -- 172.21.15.102:0/3726257279 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555966b1b540 con 0x555966c350a0 2024-04-24T23:16:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc29effd640 10 client.15687 handle_command_reply: tid=0 2024-04-24T23:16:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 client.15687 shutdown 2024-04-24T23:16:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 -- 172.21.15.102:0/3726257279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bafb60 msgr2=0x555966bb39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bafb60 0x555966bb39a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400bcb0 tx=0x7fc2b400bfd0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 -- 172.21.15.102:0/3726257279 shutdown_connections 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555966bba230 0x555966bb3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555966c350a0 0x555966c374c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555966bafb60 0x555966bb39a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc29effd640 0 client.15687 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 --2- 172.21.15.102:0/3726257279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555966baf240 0x555966bc6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 -- 172.21.15.102:0/3726257279 >> 172.21.15.102:0/3726257279 conn(0x555966bab560 msgr2=0x555966bb70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 -- 172.21.15.102:0/3726257279 shutdown_connections 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 1 -- 172.21.15.102:0/3726257279 wait complete. 2024-04-24T23:16:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.910+0000 7fc2c8a52740 20 client.15687 trim_cache size 0 max 16384 2024-04-24T23:16:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:16:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:16:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:16:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:16:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 -- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc2a80a2da0 msgr2=0x7fc2a80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc2a80a2da0 0x7fc2a80a5260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8033040 tx=0x7fc2b802a8e0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 -- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0100670 msgr2=0x7fc2c0197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0100670 0x7fc2c0197e50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000ea60 tx=0x7fc2b000ef20 comp rx=0 tx=0).stop 2024-04-24T23:16:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 -- 172.21.15.102:0/850582602 shutdown_connections 2024-04-24T23:16:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c0106630 0x7fc2c019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2c0101030 0x7fc2c0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc2a80a2da0 0x7fc2a80a5260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 --2- 172.21.15.102:0/850582602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c0100670 0x7fc2c0197e50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 -- 172.21.15.102:0/850582602 >> 172.21.15.102:0/850582602 conn(0x7fc2c00777c0 msgr2=0x7fc2c00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 -- 172.21.15.102:0/850582602 shutdown_connections 2024-04-24T23:16:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:34.911+0000 7fc2c5d42640 1 -- 172.21.15.102:0/850582602 wait complete. 2024-04-24T23:16:35.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:34 smithi183 ceph-mon[33137]: pgmap v109: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:16:35.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:34 smithi102 ceph-mon[29216]: pgmap v109: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:16:35.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:34 smithi161 ceph-mon[32709]: pgmap v109: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:16:37.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:36 smithi183 ceph-mon[33137]: pgmap v110: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:16:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:36 smithi102 ceph-mon[29216]: pgmap v110: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:16:37.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:36 smithi161 ceph-mon[32709]: pgmap v110: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:16:39.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:38 smithi183 ceph-mon[33137]: pgmap v111: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:39.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:38 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:16:39.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:38 smithi102 ceph-mon[29216]: pgmap v111: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:39.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:38 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:16:39.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:38 smithi161 ceph-mon[32709]: pgmap v111: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:39.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:38 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:16:41.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:40 smithi183 ceph-mon[33137]: pgmap v112: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:41.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:40 smithi102 ceph-mon[29216]: pgmap v112: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:41.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:40 smithi161 ceph-mon[32709]: pgmap v112: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:43.013 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:42 smithi183 ceph-mon[33137]: pgmap v113: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:43.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:42 smithi102 ceph-mon[29216]: pgmap v113: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:43.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:42 smithi161 ceph-mon[32709]: pgmap v113: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:45.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:44 smithi183 ceph-mon[33137]: pgmap v114: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:16:45.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:44 smithi102 ceph-mon[29216]: pgmap v114: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:16:45.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:44 smithi161 ceph-mon[32709]: pgmap v114: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:16:47.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:46 smithi183 ceph-mon[33137]: pgmap v115: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:47.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:46 smithi102 ceph-mon[29216]: pgmap v115: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:47.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:46 smithi161 ceph-mon[32709]: pgmap v115: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.006+0000 7f601c0a7640 1 -- 172.21.15.102:0/54533369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014100670 msgr2=0x7f6014100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.006+0000 7f601c0a7640 1 --2- 172.21.15.102:0/54533369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014100670 0x7f6014100af0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6004009930 tx=0x7f600402a420 comp rx=0 tx=0).stop 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.006+0000 7f601c0a7640 1 -- 172.21.15.102:0/54533369 shutdown_connections 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.006+0000 7f601c0a7640 1 --2- 172.21.15.102:0/54533369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6014101030 0x7f601410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.006+0000 7f601c0a7640 1 --2- 172.21.15.102:0/54533369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014100670 0x7f6014100af0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.006+0000 7f601c0a7640 1 --2- 172.21.15.102:0/54533369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6014106630 0x7f6014106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.006+0000 7f601c0a7640 1 -- 172.21.15.102:0/54533369 >> 172.21.15.102:0/54533369 conn(0x7f60140777c0 msgr2=0x7f6014077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 -- 172.21.15.102:0/54533369 shutdown_connections 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 -- 172.21.15.102:0/54533369 wait complete. 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 Processor -- start 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 -- start start 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6014100670 0x7f6014197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014101030 0x7f6014198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6014106630 0x7f601419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f60141108f0 con 0x7f6014101030 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6014110770 con 0x7f6014100670 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f601c0a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6014110a70 con 0x7f6014106630 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.007+0000 7f6019642640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014101030 0x7f6014198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f6019642640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014101030 0x7f6014198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37402/0 (socket says 172.21.15.102:37402) 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f6019642640 1 -- 172.21.15.102:0/2213532631 learned_addr learned my addr 172.21.15.102:0/2213532631 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f6019e43640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6014100670 0x7f6014197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f601a644640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6014106630 0x7f601419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f6019642640 1 -- 172.21.15.102:0/2213532631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6014106630 msgr2=0x7f601419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f6019642640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6014106630 0x7f601419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f6019642640 1 -- 172.21.15.102:0/2213532631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6014100670 msgr2=0x7f6014197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f6019642640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6014100670 0x7f6014197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f6019642640 1 -- 172.21.15.102:0/2213532631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601419cf10 con 0x7f6014101030 2024-04-24T23:16:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f6019642640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014101030 0x7f6014198460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6004005940 tx=0x7f60040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f600affd640 1 -- 172.21.15.102:0/2213532631 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600403f030 con 0x7f6014101030 2024-04-24T23:16:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f601c0a7640 1 -- 172.21.15.102:0/2213532631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601419d1a0 con 0x7f6014101030 2024-04-24T23:16:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f600affd640 1 -- 172.21.15.102:0/2213532631 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6004003cb0 con 0x7f6014101030 2024-04-24T23:16:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f600affd640 1 -- 172.21.15.102:0/2213532631 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004035070 con 0x7f6014101030 2024-04-24T23:16:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.008+0000 7f601c0a7640 1 -- 172.21.15.102:0/2213532631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60141a4a90 con 0x7f6014101030 2024-04-24T23:16:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f600affd640 1 -- 172.21.15.102:0/2213532631 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6004042030 con 0x7f6014101030 2024-04-24T23:16:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f601edb0740 1 Processor -- start 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f601edb0740 1 -- start start 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f601edb0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0e7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f601edb0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f601edb0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 0x5580be0f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f600affd640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f60000a2da0 0x7f60000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f601edb0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5580be04bcb0 con 0x5580be0e7240 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f601edb0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5580bdff45b0 con 0x5580be0f2230 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.011+0000 7f601edb0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5580be047c70 con 0x5580be0e7b60 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0e7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37410/0 (socket says 172.21.15.102:37410) 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 -- 172.21.15.102:0/1895778335 learned_addr learned my addr 172.21.15.102:0/1895778335 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f60000a2da0 0x7f60000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f600affd640 1 -- 172.21.15.102:0/2213532631 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f60040efa20 con 0x7f6014101030 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f6019642640 1 --2- 172.21.15.102:0/1895778335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 0x5580be0f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f6019e43640 1 --2- 172.21.15.102:0/1895778335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 -- 172.21.15.102:0/1895778335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 msgr2=0x5580be0f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 --2- 172.21.15.102:0/1895778335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 -- 172.21.15.102:0/1895778335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 msgr2=0x5580be0f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 --2- 172.21.15.102:0/1895778335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 0x5580be0f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 -- 172.21.15.102:0/1895778335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580bdd60210 con 0x5580be0e7240 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 --2- 172.21.15.102:0/1895778335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0e7620 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f600c014c30 tx=0x7f600c016720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888057812221fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f601a644640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f60000a2da0 0x7f60000a5260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f600c006d20 tx=0x7f600c01a000 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f5ffffff640 1 -- 172.21.15.102:0/1895778335 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c01c070 con 0x5580be0e7240 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f5ffffff640 1 -- 172.21.15.102:0/1895778335 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f600c0067e0 con 0x5580be0e7240 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.012+0000 7f5ffffff640 1 -- 172.21.15.102:0/1895778335 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0173c0 con 0x5580be0e7240 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.013+0000 7f601edb0740 1 -- 172.21.15.102:0/1895778335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 msgr2=0x5580be0e7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.013+0000 7f601edb0740 1 --2- 172.21.15.102:0/1895778335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0e7620 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f600c014c30 tx=0x7f600c016720 comp rx=0 tx=0).stop 2024-04-24T23:16:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.013+0000 7f601edb0740 1 -- 172.21.15.102:0/1895778335 shutdown_connections 2024-04-24T23:16:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.013+0000 7f601edb0740 1 --2- 172.21.15.102:0/1895778335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 0x5580be0f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.013+0000 7f601edb0740 1 --2- 172.21.15.102:0/1895778335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.013+0000 7f601edb0740 1 --2- 172.21.15.102:0/1895778335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0e7620 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.013+0000 7f601edb0740 1 -- 172.21.15.102:0/1895778335 >> 172.21.15.102:0/1895778335 conn(0x5580be0e3560 msgr2=0x5580be0e3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.013+0000 7f601edb0740 1 -- 172.21.15.102:0/1895778335 shutdown_connections 2024-04-24T23:16:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.013+0000 7f601edb0740 1 -- 172.21.15.102:0/1895778335 wait complete. 2024-04-24T23:16:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601edb0740 1 -- start start 2024-04-24T23:16:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601edb0740 1 Processor -- start 2024-04-24T23:16:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601edb0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 0x5580be0ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601edb0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0ef730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601edb0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0f2230 0x5580be0efc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601edb0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5580be0f4f60 con 0x5580be0f2230 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601edb0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5580be007b50 con 0x5580be0e7240 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601edb0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5580be0f50e0 con 0x5580be0e7b60 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601edb0740 10 client.0 _ll_register_callbacks cb 0x5580be042ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601a644640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0f2230 0x5580be0efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601a644640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0f2230 0x5580be0efc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37422/0 (socket says 172.21.15.102:37422) 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601a644640 1 -- 172.21.15.102:0/2923203438 learned_addr learned my addr 172.21.15.102:0/2923203438 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f6019e43640 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 0x5580be0ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f6019642640 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.014+0000 7f601a644640 1 -- 172.21.15.102:0/2923203438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 msgr2=0x5580be0ef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f601a644640 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0ef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f601a644640 1 -- 172.21.15.102:0/2923203438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 msgr2=0x5580be0ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f601a644640 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 0x5580be0ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f601a644640 1 -- 172.21.15.102:0/2923203438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580bde2c620 con 0x5580be0f2230 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f601a644640 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0f2230 0x5580be0efc70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f600c004b90 tx=0x7f600c018ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bef90afca4b5830 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f5ffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2923203438 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c02f070 con 0x5580be0f2230 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2923203438 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f600c020610 con 0x5580be0f2230 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f601edb0740 1 -- 172.21.15.102:0/2923203438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600c0067e0 con 0x5580be0f2230 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2923203438 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f600c03b050 con 0x5580be0f2230 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f601edb0740 10 client.15699 fetch_fsmap learned FSMap version 24 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f601edb0740 1 -- 172.21.15.102:0/2923203438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5580be1011a0 con 0x5580be0f2230 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2923203438 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f600c0067e0 con 0x5580be0f2230 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.015+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2923203438 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f600c02ba00 con 0x5580be0f2230 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.016+0000 7f601edb0740 10 client.15699 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.016+0000 7f601edb0740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.016+0000 7f601edb0740 20 client.15699 populate_metadata read hostname 'smithi102' 2024-04-24T23:16:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.016+0000 7f601edb0740 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5580be102610 0x5580be104a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.016+0000 7f601edb0740 4 client.15699 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.016+0000 7f601edb0740 1 -- 172.21.15.102:0/2923203438 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580be0591f0 con 0x5580be102610 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.016+0000 7f6019e43640 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5580be102610 0x5580be104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.016+0000 7f5ffd7fa640 10 client.15699 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.016+0000 7f6019e43640 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5580be102610 0x5580be104a00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8999afb56bea29 server_cookie=f6127e0f97de2d86 in_seq=0 out_seq=0 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2923203438 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580be0591f0 con 0x5580be102610 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f5ffd7fa640 10 client.15699 handle_command_reply: tid=0 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 client.15699 shutdown 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 -- 172.21.15.102:0/2923203438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0f2230 msgr2=0x5580be0efc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0f2230 0x5580be0efc70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f600c004b90 tx=0x7f600c018ec0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 -- 172.21.15.102:0/2923203438 shutdown_connections 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5580be102610 0x5580be104a00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0f2230 0x5580be0efc70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f5ffd7fa640 0 client.15699 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 --2- 172.21.15.102:0/2923203438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 0x5580be0ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 -- 172.21.15.102:0/2923203438 >> 172.21.15.102:0/2923203438 conn(0x5580be0e3560 msgr2=0x5580be0e9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 -- 172.21.15.102:0/2923203438 shutdown_connections 2024-04-24T23:16:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.017+0000 7f601edb0740 1 -- 172.21.15.102:0/2923203438 wait complete. 2024-04-24T23:16:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.018+0000 7f601edb0740 20 client.15699 trim_cache size 0 max 16384 2024-04-24T23:16:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.037+0000 7f601edb0740 1 Processor -- start 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.037+0000 7f601edb0740 1 -- start start 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f601edb0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 0x5580be0fde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f601edb0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7b60 0x5580be0e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f601edb0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0f2230 0x5580be10a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f601edb0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f60141afc40 con 0x5580be0e7b60 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f601edb0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f60141ad510 con 0x5580be0e7240 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f601edb0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60141ab380 con 0x5580be0f2230 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019642640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7b60 0x5580be0e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019642640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7b60 0x5580be0e3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37424/0 (socket says 172.21.15.102:37424) 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019642640 1 -- 172.21.15.102:0/71176899 learned_addr learned my addr 172.21.15.102:0/71176899 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f601a644640 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0f2230 0x5580be10a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019e43640 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 0x5580be0fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019642640 1 -- 172.21.15.102:0/71176899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0f2230 msgr2=0x5580be10a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019642640 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0f2230 0x5580be10a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019642640 1 -- 172.21.15.102:0/71176899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 msgr2=0x5580be0fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019642640 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 0x5580be0fde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019642640 1 -- 172.21.15.102:0/71176899 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580be0ee1c0 con 0x5580be0e7b60 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019e43640 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 0x5580be0fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f601a644640 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0f2230 0x5580be10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f6019642640 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7b60 0x5580be0e3940 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f60040ed760 tx=0x7f60040bba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56711220d8f77fce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f5fff7fe640 1 -- 172.21.15.102:0/71176899 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004049040 con 0x5580be0e7b60 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f5fff7fe640 1 -- 172.21.15.102:0/71176899 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60040bc630 con 0x5580be0e7b60 2024-04-24T23:16:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.038+0000 7f5fff7fe640 1 -- 172.21.15.102:0/71176899 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60040c4ba0 con 0x5580be0e7b60 2024-04-24T23:16:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.039+0000 7f601edb0740 1 -- 172.21.15.102:0/71176899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7b60 msgr2=0x5580be0e3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.039+0000 7f601edb0740 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7b60 0x5580be0e3940 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f60040ed760 tx=0x7f60040bba70 comp rx=0 tx=0).stop 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.039+0000 7f601edb0740 1 -- 172.21.15.102:0/71176899 shutdown_connections 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.039+0000 7f601edb0740 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0f2230 0x5580be10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.039+0000 7f601edb0740 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7b60 0x5580be0e3940 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.039+0000 7f601edb0740 1 --2- 172.21.15.102:0/71176899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0e7240 0x5580be0fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.039+0000 7f601edb0740 1 -- 172.21.15.102:0/71176899 >> 172.21.15.102:0/71176899 conn(0x5580be0e3560 msgr2=0x5580be0e25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.039+0000 7f601edb0740 1 -- 172.21.15.102:0/71176899 shutdown_connections 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.039+0000 7f601edb0740 1 -- 172.21.15.102:0/71176899 wait complete. 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601edb0740 1 -- start start 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601edb0740 1 Processor -- start 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601edb0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0ec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601edb0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601edb0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 0x5580be167990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601edb0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f60141ab9d0 con 0x5580be0e7240 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601edb0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f60141ace40 con 0x5580be0f2230 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601edb0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60141ac0a0 con 0x5580be0e7b60 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601edb0740 10 client.0 _ll_register_callbacks cb 0x7f600c008070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f6019e43640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f6019e43640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0ec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37436/0 (socket says 172.21.15.102:37436) 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f6019e43640 1 -- 172.21.15.102:0/2556638758 learned_addr learned my addr 172.21.15.102:0/2556638758 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f6019642640 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f601a644640 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 0x5580be167990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f6019e43640 1 -- 172.21.15.102:0/2556638758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 msgr2=0x5580be0ec7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f6019e43640 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0ec7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f6019e43640 1 -- 172.21.15.102:0/2556638758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 msgr2=0x5580be167990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f6019e43640 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 0x5580be167990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.040+0000 7f6019e43640 1 -- 172.21.15.102:0/2556638758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580be167ed0 con 0x5580be0e7240 2024-04-24T23:16:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f6019e43640 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0ec290 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f60100058c0 tx=0x7f60100118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb0c25ba88fc04d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f5ffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2556638758 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010021070 con 0x5580be0e7240 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2556638758 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f601001d880 con 0x5580be0e7240 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f601edb0740 1 -- 172.21.15.102:0/2556638758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60040bc630 con 0x5580be0e7240 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2556638758 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f601002d050 con 0x5580be0e7240 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f601edb0740 10 client.15705 fetch_fsmap learned FSMap version 24 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f601edb0740 1 -- 172.21.15.102:0/2556638758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5580be169030 con 0x5580be0e7240 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2556638758 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60040bc630 con 0x5580be0e7240 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2556638758 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6010025570 con 0x5580be0e7240 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f601edb0740 10 client.15705 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f601edb0740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f601edb0740 20 client.15705 populate_metadata read hostname 'smithi102' 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f601edb0740 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5580be102610 0x5580be16b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f601edb0740 4 client.15705 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:16:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.041+0000 7f601edb0740 1 -- 172.21.15.102:0/2556638758 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580be053540 con 0x5580be102610 2024-04-24T23:16:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.042+0000 7f6019642640 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5580be102610 0x5580be16b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.042+0000 7f5ffd7fa640 10 client.15705 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.042+0000 7f6019642640 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5580be102610 0x5580be16b850 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e58756551a503125 server_cookie=fd54f510b3fa5fe8 in_seq=0 out_seq=0 2024-04-24T23:16:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f5ffd7fa640 1 -- 172.21.15.102:0/2556638758 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580be053540 con 0x5580be102610 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f5ffd7fa640 10 client.15705 handle_command_reply: tid=0 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 client.15705 shutdown 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 -- 172.21.15.102:0/2556638758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 msgr2=0x5580be0ec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0ec290 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f60100058c0 tx=0x7f60100118d0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 -- 172.21.15.102:0/2556638758 shutdown_connections 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5580be0f2230 0x5580be167990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580be0e7b60 0x5580be0ec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5580be102610 0x5580be16b850 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 --2- 172.21.15.102:0/2556638758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5580be0e7240 0x5580be0ec290 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f5ffd7fa640 0 client.15705 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 -- 172.21.15.102:0/2556638758 >> 172.21.15.102:0/2556638758 conn(0x5580be0e3560 msgr2=0x5580be0e25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 -- 172.21.15.102:0/2556638758 shutdown_connections 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 1 -- 172.21.15.102:0/2556638758 wait complete. 2024-04-24T23:16:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.050+0000 7f601edb0740 20 client.15705 trim_cache size 0 max 16384 2024-04-24T23:16:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:16:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:16:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:16:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 -- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f60000a2da0 msgr2=0x7f60000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f60000a2da0 0x7f60000a5260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f600c006d20 tx=0x7f600c01a000 comp rx=0 tx=0).stop 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 -- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014101030 msgr2=0x7f6014198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014101030 0x7f6014198460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6004005940 tx=0x7f60040042f0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 -- 172.21.15.102:0/2213532631 shutdown_connections 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6014106630 0x7f601419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f60000a2da0 0x7f60000a5260 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6014101030 0x7f6014198460 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 --2- 172.21.15.102:0/2213532631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6014100670 0x7f6014197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 -- 172.21.15.102:0/2213532631 >> 172.21.15.102:0/2213532631 conn(0x7f60140777c0 msgr2=0x7f6014069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 -- 172.21.15.102:0/2213532631 shutdown_connections 2024-04-24T23:16:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:16:48.051+0000 7f601c0a7640 1 -- 172.21.15.102:0/2213532631 wait complete. 2024-04-24T23:16:49.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:48 smithi183 ceph-mon[33137]: pgmap v116: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:49.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:48 smithi102 ceph-mon[29216]: pgmap v116: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:49.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:48 smithi161 ceph-mon[32709]: pgmap v116: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:51.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:50 smithi183 ceph-mon[33137]: pgmap v117: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:16:51.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:50 smithi102 ceph-mon[29216]: pgmap v117: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:16:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:50 smithi161 ceph-mon[32709]: pgmap v117: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:16:53.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:52 smithi183 ceph-mon[33137]: pgmap v118: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:16:53.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:52 smithi102 ceph-mon[29216]: pgmap v118: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:16:53.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:52 smithi161 ceph-mon[32709]: pgmap v118: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:16:55.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:54 smithi183 ceph-mon[33137]: pgmap v119: 97 pgs: 97 active+clean; 4.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:16:55.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:54 smithi102 ceph-mon[29216]: pgmap v119: 97 pgs: 97 active+clean; 4.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:16:55.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:54 smithi161 ceph-mon[32709]: pgmap v119: 97 pgs: 97 active+clean; 4.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:16:57.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:56 smithi183 ceph-mon[33137]: pgmap v120: 97 pgs: 97 active+clean; 4.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:57.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:56 smithi102 ceph-mon[29216]: pgmap v120: 97 pgs: 97 active+clean; 4.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:57.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:56 smithi161 ceph-mon[32709]: pgmap v120: 97 pgs: 97 active+clean; 4.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:16:59.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:16:58 smithi183 ceph-mon[33137]: pgmap v121: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:59.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:16:58 smithi102 ceph-mon[29216]: pgmap v121: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:16:59.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:16:58 smithi161 ceph-mon[32709]: pgmap v121: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:17:01.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:00 smithi183 ceph-mon[33137]: pgmap v122: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:01.074 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:00 smithi102 ceph-mon[29216]: pgmap v122: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:01.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:00 smithi161 ceph-mon[32709]: pgmap v122: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 -- 172.21.15.102:0/3800264380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594100670 msgr2=0x7f0594100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 --2- 172.21.15.102:0/3800264380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594100670 0x7f0594100af0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0580009930 tx=0x7f058002a420 comp rx=0 tx=0).stop 2024-04-24T23:17:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 -- 172.21.15.102:0/3800264380 shutdown_connections 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 --2- 172.21.15.102:0/3800264380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0594101030 0x7f059410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 --2- 172.21.15.102:0/3800264380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594100670 0x7f0594100af0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 --2- 172.21.15.102:0/3800264380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0594106630 0x7f0594106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 -- 172.21.15.102:0/3800264380 >> 172.21.15.102:0/3800264380 conn(0x7f05940777c0 msgr2=0x7f0594077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 -- 172.21.15.102:0/3800264380 shutdown_connections 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 -- 172.21.15.102:0/3800264380 wait complete. 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.138+0000 7f059af63640 1 Processor -- start 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f059af63640 1 -- start start 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f059af63640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0594100670 0x7f059419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f059af63640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594101030 0x7f059419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f059af63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0594106630 0x7f059419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f059af63640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05941108f0 con 0x7f0594101030 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f059af63640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0594110770 con 0x7f0594106630 2024-04-24T23:17:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f059af63640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0594110a70 con 0x7f0594100670 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594101030 0x7f059419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594101030 0x7f059419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34006/0 (socket says 172.21.15.102:34006) 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0593fff640 1 -- 172.21.15.102:0/2119503948 learned_addr learned my addr 172.21.15.102:0/2119503948 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0598cff640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0594100670 0x7f059419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0599500640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0594106630 0x7f059419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0593fff640 1 -- 172.21.15.102:0/2119503948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0594100670 msgr2=0x7f059419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0593fff640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0594100670 0x7f059419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0593fff640 1 -- 172.21.15.102:0/2119503948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0594106630 msgr2=0x7f059419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0593fff640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0594106630 0x7f059419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0593fff640 1 -- 172.21.15.102:0/2119503948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059419f0d0 con 0x7f0594101030 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0598cff640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0594100670 0x7f059419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.139+0000 7f0593fff640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594101030 0x7f059419a620 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0580005940 tx=0x7f05800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.140+0000 7f0591ffb640 1 -- 172.21.15.102:0/2119503948 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058003f030 con 0x7f0594101030 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.140+0000 7f059af63640 1 -- 172.21.15.102:0/2119503948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059419f360 con 0x7f0594101030 2024-04-24T23:17:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.140+0000 7f0591ffb640 1 -- 172.21.15.102:0/2119503948 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05800318a0 con 0x7f0594101030 2024-04-24T23:17:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.140+0000 7f0591ffb640 1 -- 172.21.15.102:0/2119503948 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580035070 con 0x7f0594101030 2024-04-24T23:17:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.140+0000 7f059af63640 1 -- 172.21.15.102:0/2119503948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05941a6c50 con 0x7f0594101030 2024-04-24T23:17:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.142+0000 7f0591ffb640 1 -- 172.21.15.102:0/2119503948 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0580042030 con 0x7f0594101030 2024-04-24T23:17:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f059dc75740 1 Processor -- start 2024-04-24T23:17:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f059dc75740 1 -- start start 2024-04-24T23:17:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f059dc75740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 0x5636f3e1a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f059dc75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e24cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f059dc75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 0x5636f3e27620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f059dc75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5636f3d7ecb0 con 0x5636f3e1ab60 2024-04-24T23:17:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f059dc75740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5636f3d275b0 con 0x5636f3e1a240 2024-04-24T23:17:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f059dc75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5636f3d7ac70 con 0x5636f3e25230 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f0591ffb640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f05840a2da0 0x7f05840a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f0598cff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e24cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f0598cff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e24cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34010/0 (socket says 172.21.15.102:34010) 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f0598cff640 1 -- 172.21.15.102:0/894043402 learned_addr learned my addr 172.21.15.102:0/894043402 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f0599500640 1 --2- 172.21.15.102:0/894043402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 0x5636f3e1a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f0599500640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f05840a2da0 0x7f05840a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.143+0000 7f0593fff640 1 --2- 172.21.15.102:0/894043402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 0x5636f3e27620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f0591ffb640 1 -- 172.21.15.102:0/2119503948 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f05800efcb0 con 0x7f0594101030 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f0598cff640 1 -- 172.21.15.102:0/894043402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 msgr2=0x5636f3e27620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f0598cff640 1 --2- 172.21.15.102:0/894043402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 0x5636f3e27620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f0598cff640 1 -- 172.21.15.102:0/894043402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 msgr2=0x5636f3e1a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f0598cff640 1 --2- 172.21.15.102:0/894043402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 0x5636f3e1a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f0598cff640 1 -- 172.21.15.102:0/894043402 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f3a93210 con 0x5636f3e1ab60 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f0598cff640 1 --2- 172.21.15.102:0/894043402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e24cf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0588009cd0 tx=0x7f05880117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f986fd3e34b9a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f0599500640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f05840a2da0 0x7f05840a5260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f058c009370 tx=0x7f058c009520 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f057effd640 1 -- 172.21.15.102:0/894043402 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588012450 con 0x5636f3e1ab60 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f057effd640 1 -- 172.21.15.102:0/894043402 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0588012aa0 con 0x5636f3e1ab60 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.144+0000 7f057effd640 1 -- 172.21.15.102:0/894043402 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058800fb60 con 0x5636f3e1ab60 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 -- 172.21.15.102:0/894043402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 msgr2=0x5636f3e24cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 --2- 172.21.15.102:0/894043402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e24cf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0588009cd0 tx=0x7f05880117b0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 -- 172.21.15.102:0/894043402 shutdown_connections 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 --2- 172.21.15.102:0/894043402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 0x5636f3e27620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 --2- 172.21.15.102:0/894043402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e24cf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 --2- 172.21.15.102:0/894043402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 0x5636f3e1a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 -- 172.21.15.102:0/894043402 >> 172.21.15.102:0/894043402 conn(0x5636f3e16560 msgr2=0x5636f3e16940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 -- 172.21.15.102:0/894043402 shutdown_connections 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 -- 172.21.15.102:0/894043402 wait complete. 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 -- start start 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.145+0000 7f059dc75740 1 Processor -- start 2024-04-24T23:17:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f059dc75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f059dc75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 0x5636f3e22650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f059dc75740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 0x5636f3e22b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f059dc75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5636f3e28020 con 0x5636f3e1a240 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f059dc75740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5636f3d3ab50 con 0x5636f3e25230 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f059dc75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5636f3e281a0 con 0x5636f3e1ab60 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f059dc75740 10 client.0 _ll_register_callbacks cb 0x5636f3d75ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0598cff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0598cff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e22110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34012/0 (socket says 172.21.15.102:34012) 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0598cff640 1 -- 172.21.15.102:0/2772233203 learned_addr learned my addr 172.21.15.102:0/2772233203 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0599500640 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 0x5636f3e22b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0593fff640 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 0x5636f3e22650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0598cff640 1 -- 172.21.15.102:0/2772233203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 msgr2=0x5636f3e22650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0598cff640 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 0x5636f3e22650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0598cff640 1 -- 172.21.15.102:0/2772233203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 msgr2=0x5636f3e22b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0598cff640 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 0x5636f3e22b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0598cff640 1 -- 172.21.15.102:0/2772233203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f3b5f620 con 0x5636f3e1a240 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0599500640 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 0x5636f3e22b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0598cff640 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e22110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f058801c040 tx=0x7f0588005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4b364028843617 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0573fff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0573fff640 1 -- 172.21.15.102:0/2772233203 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588012650 con 0x5636f3e1a240 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f0573fff640 1 -- 172.21.15.102:0/2772233203 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f05880127f0 con 0x5636f3e1a240 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.146+0000 7f059dc75740 1 -- 172.21.15.102:0/2772233203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0588012aa0 con 0x5636f3e1a240 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f0573fff640 1 -- 172.21.15.102:0/2772233203 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0588032050 con 0x5636f3e1a240 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f059dc75740 10 client.15717 fetch_fsmap learned FSMap version 24 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f059dc75740 1 -- 172.21.15.102:0/2772233203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5636f3e341a0 con 0x5636f3e1a240 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f0573fff640 1 -- 172.21.15.102:0/2772233203 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0588012aa0 con 0x5636f3e1a240 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f0573fff640 1 -- 172.21.15.102:0/2772233203 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f058800fcf0 con 0x5636f3e1a240 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f059dc75740 10 client.15717 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f059dc75740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f059dc75740 20 client.15717 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f059dc75740 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5636f3e35600 0x5636f3e379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f059dc75740 4 client.15717 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f059dc75740 1 -- 172.21.15.102:0/2772233203 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636f3d8c1f0 con 0x5636f3e35600 2024-04-24T23:17:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.147+0000 7f0593fff640 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5636f3e35600 0x5636f3e379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.148+0000 7f0593fff640 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5636f3e35600 0x5636f3e379f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b95ab3b4fe4b8b6 server_cookie=f42f9b6b724ebc2f in_seq=0 out_seq=0 2024-04-24T23:17:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.148+0000 7f0573fff640 10 client.15717 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.148+0000 7f0573fff640 1 -- 172.21.15.102:0/2772233203 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636f3d8c1f0 con 0x5636f3e35600 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.148+0000 7f0573fff640 10 client.15717 handle_command_reply: tid=0 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.148+0000 7f059dc75740 1 client.15717 shutdown 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 -- 172.21.15.102:0/2772233203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 msgr2=0x5636f3e22110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e22110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f058801c040 tx=0x7f0588005730 comp rx=0 tx=0).stop 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 -- 172.21.15.102:0/2772233203 shutdown_connections 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 0x5636f3e22b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 0x5636f3e22650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5636f3e35600 0x5636f3e379f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 --2- 172.21.15.102:0/2772233203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e22110 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f0573fff640 0 client.15717 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 -- 172.21.15.102:0/2772233203 >> 172.21.15.102:0/2772233203 conn(0x5636f3e16560 msgr2=0x5636f3e1c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 -- 172.21.15.102:0/2772233203 shutdown_connections 2024-04-24T23:17:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 1 -- 172.21.15.102:0/2772233203 wait complete. 2024-04-24T23:17:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.149+0000 7f059dc75740 20 client.15717 trim_cache size 0 max 16384 2024-04-24T23:17:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.164+0000 7f059dc75740 1 Processor -- start 2024-04-24T23:17:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.164+0000 7f059dc75740 1 -- start start 2024-04-24T23:17:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.164+0000 7f059dc75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e30f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.164+0000 7f059dc75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 0x5636f3e3d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.164+0000 7f059dc75740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 0x5636f3e3d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.164+0000 7f059dc75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05941b1e60 con 0x5636f3e1a240 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.164+0000 7f059dc75740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05941af730 con 0x5636f3e25230 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.164+0000 7f059dc75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05941ad5a0 con 0x5636f3e1ab60 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.164+0000 7f0598cff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e30f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0598cff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e30f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34022/0 (socket says 172.21.15.102:34022) 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0598cff640 1 -- 172.21.15.102:0/4066325494 learned_addr learned my addr 172.21.15.102:0/4066325494 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0599500640 1 --2- 172.21.15.102:0/4066325494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 0x5636f3e3d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0593fff640 1 --2- 172.21.15.102:0/4066325494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 0x5636f3e3d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0598cff640 1 -- 172.21.15.102:0/4066325494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 msgr2=0x5636f3e3d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0598cff640 1 --2- 172.21.15.102:0/4066325494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 0x5636f3e3d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0598cff640 1 -- 172.21.15.102:0/4066325494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 msgr2=0x5636f3e3d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0598cff640 1 --2- 172.21.15.102:0/4066325494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 0x5636f3e3d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0598cff640 1 -- 172.21.15.102:0/4066325494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f3e27990 con 0x5636f3e1a240 2024-04-24T23:17:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f0598cff640 1 --2- 172.21.15.102:0/4066325494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e30f10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f058800f790 tx=0x7f0588009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ffd9af6a5c1d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f057e7fc640 1 -- 172.21.15.102:0/4066325494 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05880043c0 con 0x5636f3e1a240 2024-04-24T23:17:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f057e7fc640 1 -- 172.21.15.102:0/4066325494 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f058803a030 con 0x5636f3e1a240 2024-04-24T23:17:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.165+0000 7f057e7fc640 1 -- 172.21.15.102:0/4066325494 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588012450 con 0x5636f3e1a240 2024-04-24T23:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 -- 172.21.15.102:0/4066325494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 msgr2=0x5636f3e30f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 --2- 172.21.15.102:0/4066325494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e30f10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f058800f790 tx=0x7f0588009b20 comp rx=0 tx=0).stop 2024-04-24T23:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 -- 172.21.15.102:0/4066325494 shutdown_connections 2024-04-24T23:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 --2- 172.21.15.102:0/4066325494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e25230 0x5636f3e3d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 --2- 172.21.15.102:0/4066325494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e1ab60 0x5636f3e3d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 --2- 172.21.15.102:0/4066325494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1a240 0x5636f3e30f10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 -- 172.21.15.102:0/4066325494 >> 172.21.15.102:0/4066325494 conn(0x5636f3e16560 msgr2=0x5636f3e155c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 -- 172.21.15.102:0/4066325494 shutdown_connections 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 -- 172.21.15.102:0/4066325494 wait complete. 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 -- start start 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.166+0000 7f059dc75740 1 Processor -- start 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f059dc75740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 0x5636f3e2d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f059dc75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e1e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f059dc75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 0x5636f3e1eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f059dc75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05941adbf0 con 0x5636f3e1ab60 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f059dc75740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05941af060 con 0x5636f3e1a240 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f059dc75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05941ae2c0 con 0x5636f3e25230 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f059dc75740 10 client.0 _ll_register_callbacks cb 0x7f058c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e1e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e1e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34032/0 (socket says 172.21.15.102:34032) 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0593fff640 1 -- 172.21.15.102:0/2140213729 learned_addr learned my addr 172.21.15.102:0/2140213729 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0599500640 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 0x5636f3e1eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0598cff640 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 0x5636f3e2d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0593fff640 1 -- 172.21.15.102:0/2140213729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 msgr2=0x5636f3e1eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0593fff640 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 0x5636f3e1eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0593fff640 1 -- 172.21.15.102:0/2140213729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 msgr2=0x5636f3e2d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0593fff640 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 0x5636f3e2d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0593fff640 1 -- 172.21.15.102:0/2140213729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f3e1f4e0 con 0x5636f3e1ab60 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0598cff640 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 0x5636f3e2d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0593fff640 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e1e960 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f05800bbec0 tx=0x7f05800871e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2743f0249d86c3a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.167+0000 7f0573fff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f0573fff640 1 -- 172.21.15.102:0/2140213729 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580049040 con 0x5636f3e1ab60 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f0573fff640 1 -- 172.21.15.102:0/2140213729 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f05800ed670 con 0x5636f3e1ab60 2024-04-24T23:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f059dc75740 1 -- 172.21.15.102:0/2140213729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f058803a030 con 0x5636f3e1ab60 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f0573fff640 1 -- 172.21.15.102:0/2140213729 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f05800bc8c0 con 0x5636f3e1ab60 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f059dc75740 10 client.15723 fetch_fsmap learned FSMap version 24 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f059dc75740 1 -- 172.21.15.102:0/2140213729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5636f3e9b760 con 0x5636f3e1ab60 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f0573fff640 1 -- 172.21.15.102:0/2140213729 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f058803a030 con 0x5636f3e1ab60 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f0573fff640 1 -- 172.21.15.102:0/2140213729 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f05800b8830 con 0x5636f3e1ab60 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f059dc75740 10 client.15723 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f059dc75740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f059dc75740 20 client.15723 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f059dc75740 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5636f3e9be90 0x5636f3e9e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f059dc75740 4 client.15723 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f059dc75740 1 -- 172.21.15.102:0/2140213729 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636f3d86540 con 0x5636f3e9be90 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.168+0000 7f0598cff640 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5636f3e9be90 0x5636f3e9e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.169+0000 7f0573fff640 10 client.15723 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.169+0000 7f0598cff640 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5636f3e9be90 0x5636f3e9e2b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f2ba048f5a36a61 server_cookie=2f13c92c3baa5e4e in_seq=0 out_seq=0 2024-04-24T23:17:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f0573fff640 1 -- 172.21.15.102:0/2140213729 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636f3d86540 con 0x5636f3e9be90 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f0573fff640 10 client.15723 handle_command_reply: tid=0 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 client.15723 shutdown 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 -- 172.21.15.102:0/2140213729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 msgr2=0x5636f3e1e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e1e960 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f05800bbec0 tx=0x7f05800871e0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 -- 172.21.15.102:0/2140213729 shutdown_connections 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636f3e25230 0x5636f3e1eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5636f3e9be90 0x5636f3e9e2b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636f3e1ab60 0x5636f3e1e960 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 --2- 172.21.15.102:0/2140213729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5636f3e1a240 0x5636f3e2d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f0573fff640 0 client.15723 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 -- 172.21.15.102:0/2140213729 >> 172.21.15.102:0/2140213729 conn(0x5636f3e16560 msgr2=0x5636f3e220b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 -- 172.21.15.102:0/2140213729 shutdown_connections 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 1 -- 172.21.15.102:0/2140213729 wait complete. 2024-04-24T23:17:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.177+0000 7f059dc75740 20 client.15723 trim_cache size 0 max 16384 2024-04-24T23:17:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:17:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:17:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:17:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 -- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f05840a2da0 msgr2=0x7f05840a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f05840a2da0 0x7f05840a5260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f058c009370 tx=0x7f058c009520 comp rx=0 tx=0).stop 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 -- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594101030 msgr2=0x7f059419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594101030 0x7f059419a620 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0580005940 tx=0x7f05800042f0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 -- 172.21.15.102:0/2119503948 shutdown_connections 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0594106630 0x7f059419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f05840a2da0 0x7f05840a5260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0594101030 0x7f059419a620 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 --2- 172.21.15.102:0/2119503948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0594100670 0x7f059419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 -- 172.21.15.102:0/2119503948 >> 172.21.15.102:0/2119503948 conn(0x7f05940777c0 msgr2=0x7f0594068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 -- 172.21.15.102:0/2119503948 shutdown_connections 2024-04-24T23:17:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:01.178+0000 7f059af63640 1 -- 172.21.15.102:0/2119503948 wait complete. 2024-04-24T23:17:03.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:02 smithi183 ceph-mon[33137]: pgmap v123: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:03.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:02 smithi102 ceph-mon[29216]: pgmap v123: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:03.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:02 smithi161 ceph-mon[32709]: pgmap v123: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:05.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:04 smithi183 ceph-mon[33137]: pgmap v124: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 42 op/s 2024-04-24T23:17:05.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:04 smithi102 ceph-mon[29216]: pgmap v124: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 42 op/s 2024-04-24T23:17:05.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:04 smithi161 ceph-mon[32709]: pgmap v124: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 42 op/s 2024-04-24T23:17:07.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:06 smithi183 ceph-mon[33137]: pgmap v125: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:17:07.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:06 smithi102 ceph-mon[29216]: pgmap v125: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:17:07.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:06 smithi161 ceph-mon[32709]: pgmap v125: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:17:09.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:08 smithi183 ceph-mon[33137]: pgmap v126: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 39 op/s 2024-04-24T23:17:09.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:08 smithi102 ceph-mon[29216]: pgmap v126: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 39 op/s 2024-04-24T23:17:09.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:08 smithi161 ceph-mon[32709]: pgmap v126: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 39 op/s 2024-04-24T23:17:11.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:10 smithi183 ceph-mon[33137]: pgmap v127: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:17:11.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:10 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:11.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:10 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:11.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:10 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:11.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:10 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:17:11.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:10 smithi102 ceph-mon[29216]: pgmap v127: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:17:11.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:10 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:11.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:10 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:11.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:10 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:11.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:10 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:17:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:10 smithi161 ceph-mon[32709]: pgmap v127: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:17:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:10 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:10 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:10 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:11.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:10 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:17:12.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:11 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:17:12.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:11 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:17:12.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:11 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:17:13.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:12 smithi102 ceph-mon[29216]: pgmap v128: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:17:13.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:12 smithi161 ceph-mon[32709]: pgmap v128: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:17:13.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:12 smithi183 ceph-mon[33137]: pgmap v128: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:17:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 -- 172.21.15.102:0/4116358897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 msgr2=0x7fcaa410d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/4116358897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 0x7fcaa410d4f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009ca0 tx=0x7fca9c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 -- 172.21.15.102:0/4116358897 shutdown_connections 2024-04-24T23:17:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/4116358897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 0x7fcaa410d4f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/4116358897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa4100620 0x7fcaa4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/4116358897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa4106630 0x7fcaa4106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 -- 172.21.15.102:0/4116358897 >> 172.21.15.102:0/4116358897 conn(0x7fcaa40fc110 msgr2=0x7fcaa40fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 -- 172.21.15.102:0/4116358897 shutdown_connections 2024-04-24T23:17:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 -- 172.21.15.102:0/4116358897 wait complete. 2024-04-24T23:17:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 Processor -- start 2024-04-24T23:17:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.269+0000 7fcaaab7e640 1 -- start start 2024-04-24T23:17:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaaab7e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa4100620 0x7fcaa4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaaab7e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 0x7fcaa4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaaab7e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa4106630 0x7fcaa419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaaab7e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa4111ac0 con 0x7fcaa4100fe0 2024-04-24T23:17:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaaab7e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa4111940 con 0x7fcaa4100620 2024-04-24T23:17:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaaab7e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa4111c40 con 0x7fcaa4106630 2024-04-24T23:17:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 0x7fcaa4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 0x7fcaa4198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43636/0 (socket says 172.21.15.102:43636) 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa3fff640 1 -- 172.21.15.102:0/3373294660 learned_addr learned my addr 172.21.15.102:0/3373294660 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa911b640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa4106630 0x7fcaa419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa3fff640 1 -- 172.21.15.102:0/3373294660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa4106630 msgr2=0x7fcaa419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa4106630 0x7fcaa419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa3fff640 1 -- 172.21.15.102:0/3373294660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa4100620 msgr2=0x7fcaa4197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa4100620 0x7fcaa4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa3fff640 1 -- 172.21.15.102:0/3373294660 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa419ced0 con 0x7fcaa4100fe0 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 0x7fcaa4198420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b4a0 tx=0x7fca9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaa1ffb640 1 -- 172.21.15.102:0/3373294660 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90014070 con 0x7fcaa4100fe0 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.270+0000 7fcaaab7e640 1 -- 172.21.15.102:0/3373294660 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa419d1c0 con 0x7fcaa4100fe0 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.271+0000 7fcaa1ffb640 1 -- 172.21.15.102:0/3373294660 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca90010ac0 con 0x7fcaa4100fe0 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.271+0000 7fcaa1ffb640 1 -- 172.21.15.102:0/3373294660 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90017c30 con 0x7fcaa4100fe0 2024-04-24T23:17:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.271+0000 7fcaaab7e640 1 -- 172.21.15.102:0/3373294660 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa41a4a50 con 0x7fcaa4100fe0 2024-04-24T23:17:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.273+0000 7fcaa1ffb640 1 -- 172.21.15.102:0/3373294660 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fca90017070 con 0x7fcaa4100fe0 2024-04-24T23:17:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.273+0000 7fcaad96c740 1 Processor -- start 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.273+0000 7fcaad96c740 1 -- start start 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18120 0x55f503a18500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18a40 0x55f503a27100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a29a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaad96c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f50397ccb0 con 0x55f503a27640 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaad96c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f5039255b0 con 0x55f503a18a40 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaad96c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f503978c70 con 0x55f503a18120 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a29a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a29a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43652/0 (socket says 172.21.15.102:43652) 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa3fff640 1 -- 172.21.15.102:0/20414655 learned_addr learned my addr 172.21.15.102:0/20414655 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa891a640 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18a40 0x55f503a27100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa1ffb640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fca940a2da0 0x7fca940a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa911b640 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18120 0x55f503a18500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa911b640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fca940a2da0 0x7fca940a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa3fff640 1 -- 172.21.15.102:0/20414655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18120 msgr2=0x55f503a18500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18120 0x55f503a18500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa3fff640 1 -- 172.21.15.102:0/20414655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18a40 msgr2=0x55f503a27100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18a40 0x55f503a27100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa3fff640 1 -- 172.21.15.102:0/20414655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f503691210 con 0x55f503a27640 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.274+0000 7fcaa891a640 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18a40 0x55f503a27100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.275+0000 7fcaa1ffb640 1 -- 172.21.15.102:0/3373294660 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fca900ceac0 con 0x7fcaa4100fe0 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.275+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a29a30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fca900cc2b0 tx=0x7fca9006a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df26a6cb4889ceb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.275+0000 7fcaa911b640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fca940a2da0 0x7fca940a5260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fca9c006010 tx=0x7fca9c02a8c0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.275+0000 7fca8effd640 1 -- 172.21.15.102:0/20414655 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90001d10 con 0x55f503a27640 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.275+0000 7fca8effd640 1 -- 172.21.15.102:0/20414655 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca9009b7e0 con 0x55f503a27640 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.275+0000 7fca8effd640 1 -- 172.21.15.102:0/20414655 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9009f490 con 0x55f503a27640 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 -- 172.21.15.102:0/20414655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 msgr2=0x55f503a29a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a29a30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fca900cc2b0 tx=0x7fca9006a010 comp rx=0 tx=0).stop 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 -- 172.21.15.102:0/20414655 shutdown_connections 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a29a30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18a40 0x55f503a27100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 --2- 172.21.15.102:0/20414655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18120 0x55f503a18500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 -- 172.21.15.102:0/20414655 >> 172.21.15.102:0/20414655 conn(0x55f503a14560 msgr2=0x55f503a14940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 -- 172.21.15.102:0/20414655 shutdown_connections 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 -- 172.21.15.102:0/20414655 wait complete. 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.276+0000 7fcaad96c740 1 -- start start 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaad96c740 1 Processor -- start 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18120 0x55f503a244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 0x55f503a24a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a24f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaad96c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f503a2a420 con 0x55f503a27640 2024-04-24T23:17:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaad96c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f503938b50 con 0x55f503a18120 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaad96c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f503a2a5a0 con 0x55f503a18a40 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaad96c740 10 client.0 _ll_register_callbacks cb 0x55f503973ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaa911b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a24f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaa891a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18120 0x55f503a244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaa891a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18120 0x55f503a244c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:59486/0 (socket says 172.21.15.102:59486) 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaa891a640 1 -- 172.21.15.102:0/838164882 learned_addr learned my addr 172.21.15.102:0/838164882 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaa911b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a24f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43656/0 (socket says 172.21.15.102:43656) 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.277+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 0x55f503a24a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaa891a640 1 -- 172.21.15.102:0/838164882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 msgr2=0x55f503a24a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaa891a640 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 0x55f503a24a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaa891a640 1 -- 172.21.15.102:0/838164882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 msgr2=0x55f503a24f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaa891a640 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a24f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaa891a640 1 -- 172.21.15.102:0/838164882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50375d620 con 0x55f503a18120 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaa911b640 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a24f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaa891a640 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18120 0x55f503a244c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d210 tx=0x7fca9800d6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e85f6a70ff105828 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fca83fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fca83fff640 1 -- 172.21.15.102:0/838164882 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98012630 con 0x55f503a18120 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaad96c740 1 -- 172.21.15.102:0/838164882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9009b7e0 con 0x55f503a18120 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fca83fff640 1 -- 172.21.15.102:0/838164882 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fca980127d0 con 0x55f503a18120 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fca83fff640 1 -- 172.21.15.102:0/838164882 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca98012a10 con 0x55f503a18120 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaad96c740 10 client.25181 fetch_fsmap learned FSMap version 25 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.278+0000 7fcaad96c740 1 -- 172.21.15.102:0/838164882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f503a36630 con 0x55f503a18120 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.279+0000 7fca83fff640 1 -- 172.21.15.102:0/838164882 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca9009b7e0 con 0x55f503a18120 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.279+0000 7fca83fff640 1 -- 172.21.15.102:0/838164882 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) ==== 8342+0+0 (secure 0 0 0) 0x7fca9801c030 con 0x55f503a18120 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.279+0000 7fcaad96c740 10 client.25181 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.279+0000 7fcaad96c740 10 client.25181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.279+0000 7fcaad96c740 20 client.25181 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.279+0000 7fcaad96c740 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f503a37af0 0x55f503a39ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.279+0000 7fcaad96c740 4 client.25181 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.279+0000 7fcaad96c740 1 -- 172.21.15.102:0/838164882 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f50398a1f0 con 0x55f503a37af0 2024-04-24T23:17:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.279+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f503a37af0 0x55f503a39ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f503a37af0 0x55f503a39ee0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df369bff77a04bce server_cookie=45ad27e0bf76b676 in_seq=0 out_seq=0 2024-04-24T23:17:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fca83fff640 10 client.25181 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fca83fff640 1 -- 172.21.15.102:0/838164882 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f50398a1f0 con 0x55f503a37af0 2024-04-24T23:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fca83fff640 10 client.25181 handle_command_reply: tid=0 2024-04-24T23:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 client.25181 shutdown 2024-04-24T23:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 -- 172.21.15.102:0/838164882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18120 msgr2=0x55f503a244c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18120 0x55f503a244c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d210 tx=0x7fca9800d6d0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 -- 172.21.15.102:0/838164882 shutdown_connections 2024-04-24T23:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f503a37af0 0x55f503a39ee0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a24f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 0x55f503a24a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fca83fff640 0 client.25181 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 --2- 172.21.15.102:0/838164882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a18120 0x55f503a244c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 -- 172.21.15.102:0/838164882 >> 172.21.15.102:0/838164882 conn(0x55f503a14560 msgr2=0x55f503a1a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.280+0000 7fcaad96c740 1 -- 172.21.15.102:0/838164882 shutdown_connections 2024-04-24T23:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.281+0000 7fcaad96c740 1 -- 172.21.15.102:0/838164882 wait complete. 2024-04-24T23:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.281+0000 7fcaad96c740 20 client.25181 trim_cache size 0 max 16384 2024-04-24T23:17:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.295+0000 7fcaad96c740 1 Processor -- start 2024-04-24T23:17:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.295+0000 7fcaad96c740 1 -- start start 2024-04-24T23:17:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.295+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 0x55f503a26bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.295+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a33350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.295+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 0x55f503a13440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.295+0000 7fcaad96c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41adac0 con 0x55f503a27640 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.295+0000 7fcaad96c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41ab390 con 0x55f503a23620 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.295+0000 7fcaad96c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41a9200 con 0x55f503a18a40 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.295+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a33350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a33350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43664/0 (socket says 172.21.15.102:43664) 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa3fff640 1 -- 172.21.15.102:0/392001830 learned_addr learned my addr 172.21.15.102:0/392001830 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa911b640 1 --2- 172.21.15.102:0/392001830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 0x55f503a13440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa891a640 1 --2- 172.21.15.102:0/392001830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 0x55f503a26bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa3fff640 1 -- 172.21.15.102:0/392001830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 msgr2=0x55f503a26bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/392001830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 0x55f503a26bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa3fff640 1 -- 172.21.15.102:0/392001830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 msgr2=0x55f503a13440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/392001830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 0x55f503a13440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa3fff640 1 -- 172.21.15.102:0/392001830 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f503a18870 con 0x55f503a27640 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/392001830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a33350 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fca90097ab0 tx=0x7fca900d2280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa809eb2ba674bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fca8e7fc640 1 -- 172.21.15.102:0/392001830 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9001e040 con 0x55f503a27640 2024-04-24T23:17:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fca8e7fc640 1 -- 172.21.15.102:0/392001830 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca900a3d50 con 0x55f503a27640 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.296+0000 7fca8e7fc640 1 -- 172.21.15.102:0/392001830 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca900bb030 con 0x55f503a27640 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 -- 172.21.15.102:0/392001830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 msgr2=0x55f503a33350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 --2- 172.21.15.102:0/392001830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a33350 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fca90097ab0 tx=0x7fca900d2280 comp rx=0 tx=0).stop 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 -- 172.21.15.102:0/392001830 shutdown_connections 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 --2- 172.21.15.102:0/392001830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 0x55f503a13440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 --2- 172.21.15.102:0/392001830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a27640 0x55f503a33350 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 --2- 172.21.15.102:0/392001830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a18a40 0x55f503a26bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 -- 172.21.15.102:0/392001830 >> 172.21.15.102:0/392001830 conn(0x55f503a14560 msgr2=0x55f503a12d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 -- 172.21.15.102:0/392001830 shutdown_connections 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 -- 172.21.15.102:0/392001830 wait complete. 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 -- start start 2024-04-24T23:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.297+0000 7fcaad96c740 1 Processor -- start 2024-04-24T23:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a18a40 0x55f503a2fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 0x55f503a2aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaad96c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a27640 0x55f503a2b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaad96c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41a9850 con 0x55f503a18a40 2024-04-24T23:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaad96c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41aacc0 con 0x55f503a23620 2024-04-24T23:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaad96c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41a9f20 con 0x55f503a27640 2024-04-24T23:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaad96c740 10 client.0 _ll_register_callbacks cb 0x7fca9800b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa891a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a18a40 0x55f503a2fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa891a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a18a40 0x55f503a2fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43676/0 (socket says 172.21.15.102:43676) 2024-04-24T23:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa891a640 1 -- 172.21.15.102:0/553502629 learned_addr learned my addr 172.21.15.102:0/553502629 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 0x55f503a2aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa911b640 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a27640 0x55f503a2b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa891a640 1 -- 172.21.15.102:0/553502629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a27640 msgr2=0x55f503a2b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa891a640 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a27640 0x55f503a2b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa891a640 1 -- 172.21.15.102:0/553502629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 msgr2=0x55f503a2aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa891a640 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 0x55f503a2aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa891a640 1 -- 172.21.15.102:0/553502629 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f503a2bdd0 con 0x55f503a18a40 2024-04-24T23:17:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fcaa891a640 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a18a40 0x55f503a2fc80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fca9801de80 tx=0x7fca98006bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32dc66837c2ba3fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.298+0000 7fca83fff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:17:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fca83fff640 1 -- 172.21.15.102:0/553502629 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9802b060 con 0x55f503a18a40 2024-04-24T23:17:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fca83fff640 1 -- 172.21.15.102:0/553502629 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fca98012630 con 0x55f503a18a40 2024-04-24T23:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaad96c740 1 -- 172.21.15.102:0/553502629 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca900a3d50 con 0x55f503a18a40 2024-04-24T23:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fca83fff640 1 -- 172.21.15.102:0/553502629 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca98012bb0 con 0x55f503a18a40 2024-04-24T23:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaad96c740 10 client.15741 fetch_fsmap learned FSMap version 25 2024-04-24T23:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaad96c740 1 -- 172.21.15.102:0/553502629 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f503a9e0a0 con 0x55f503a18a40 2024-04-24T23:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fca83fff640 1 -- 172.21.15.102:0/553502629 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca900a3d50 con 0x55f503a18a40 2024-04-24T23:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fca83fff640 1 -- 172.21.15.102:0/553502629 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 25) ==== 8342+0+0 (secure 0 0 0) 0x7fca98025070 con 0x55f503a18a40 2024-04-24T23:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaad96c740 10 client.15741 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T23:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaad96c740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaad96c740 20 client.15741 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaad96c740 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f503a9e790 0x55f503aa0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaad96c740 4 client.15741 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaad96c740 1 -- 172.21.15.102:0/553502629 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f503984540 con 0x55f503a9e790 2024-04-24T23:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.299+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f503a9e790 0x55f503aa0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.300+0000 7fca83fff640 10 client.15741 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.300+0000 7fcaa3fff640 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f503a9e790 0x55f503aa0bb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58d1aa0bf6a0ad2c server_cookie=8e22568a2837ff3a in_seq=0 out_seq=0 2024-04-24T23:17:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fca83fff640 1 -- 172.21.15.102:0/553502629 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f503984540 con 0x55f503a9e790 2024-04-24T23:17:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fca83fff640 10 client.15741 handle_command_reply: tid=0 2024-04-24T23:17:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 client.15741 shutdown 2024-04-24T23:17:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 -- 172.21.15.102:0/553502629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a18a40 msgr2=0x55f503a2fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a18a40 0x55f503a2fc80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fca9801de80 tx=0x7fca98006bd0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 -- 172.21.15.102:0/553502629 shutdown_connections 2024-04-24T23:17:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f503a27640 0x55f503a2b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f503a23620 0x55f503a2aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55f503a9e790 0x55f503aa0bb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 --2- 172.21.15.102:0/553502629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f503a18a40 0x55f503a2fc80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 -- 172.21.15.102:0/553502629 >> 172.21.15.102:0/553502629 conn(0x55f503a14560 msgr2=0x55f503a135c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fca83fff640 0 client.15741 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 -- 172.21.15.102:0/553502629 shutdown_connections 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 1 -- 172.21.15.102:0/553502629 wait complete. 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.307+0000 7fcaad96c740 20 client.15741 trim_cache size 0 max 16384 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:17:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:17:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 -- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fca940a2da0 msgr2=0x7fca940a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fca940a2da0 0x7fca940a5260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fca9c006010 tx=0x7fca9c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 -- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 msgr2=0x7fcaa4198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 0x7fcaa4198420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b4a0 tx=0x7fca9000b960 comp rx=0 tx=0).stop 2024-04-24T23:17:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 -- 172.21.15.102:0/3373294660 shutdown_connections 2024-04-24T23:17:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa4106630 0x7fcaa419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fca940a2da0 0x7fca940a5260 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa4100fe0 0x7fcaa4198420 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 --2- 172.21.15.102:0/3373294660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa4100620 0x7fcaa4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 -- 172.21.15.102:0/3373294660 >> 172.21.15.102:0/3373294660 conn(0x7fcaa40fc110 msgr2=0x7fcaa40fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 -- 172.21.15.102:0/3373294660 shutdown_connections 2024-04-24T23:17:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:14.308+0000 7fcaaab7e640 1 -- 172.21.15.102:0/3373294660 wait complete. 2024-04-24T23:17:15.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:14 smithi102 ceph-mon[29216]: pgmap v129: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:17:15.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:14 smithi161 ceph-mon[32709]: pgmap v129: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:17:15.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:14 smithi183 ceph-mon[33137]: pgmap v129: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:17:17.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:16 smithi102 ceph-mon[29216]: pgmap v130: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:17.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:16 smithi161 ceph-mon[32709]: pgmap v130: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:16 smithi183 ceph-mon[33137]: pgmap v130: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:19.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:18 smithi102 ceph-mon[29216]: pgmap v131: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:19.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:18 smithi161 ceph-mon[32709]: pgmap v131: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:19.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:18 smithi183 ceph-mon[33137]: pgmap v131: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:21.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:20 smithi102 ceph-mon[29216]: pgmap v132: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:21.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:20 smithi161 ceph-mon[32709]: pgmap v132: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:20 smithi183 ceph-mon[33137]: pgmap v132: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:22.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:21 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:17:22.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:21 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:17:22.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:21 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:17:23.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:22 smithi102 ceph-mon[29216]: pgmap v133: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:23.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:22 smithi161 ceph-mon[32709]: pgmap v133: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:23.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:22 smithi183 ceph-mon[33137]: pgmap v133: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:25.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:24 smithi102 ceph-mon[29216]: pgmap v134: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:17:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:24 smithi161 ceph-mon[32709]: pgmap v134: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:17:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:24 smithi183 ceph-mon[33137]: pgmap v134: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:17:27.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:26 smithi102 ceph-mon[29216]: pgmap v135: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:27.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:26 smithi161 ceph-mon[32709]: pgmap v135: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:27.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:26 smithi183 ceph-mon[33137]: pgmap v135: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.403+0000 7f9dc03e7640 1 -- 172.21.15.102:0/3357925092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 msgr2=0x7f9db810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.403+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/3357925092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 0x7f9db810e600 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009930 tx=0x7f9db002a420 comp rx=0 tx=0).stop 2024-04-24T23:17:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.403+0000 7f9dc03e7640 1 -- 172.21.15.102:0/3357925092 shutdown_connections 2024-04-24T23:17:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.403+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/3357925092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 0x7f9db810e600 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.403+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/3357925092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db8101730 0x7f9db8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.403+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/3357925092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9db8107740 0x7f9db8107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.403+0000 7f9dc03e7640 1 -- 172.21.15.102:0/3357925092 >> 172.21.15.102:0/3357925092 conn(0x7f9db80fd200 msgr2=0x7f9db80ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.403+0000 7f9dc03e7640 1 -- 172.21.15.102:0/3357925092 shutdown_connections 2024-04-24T23:17:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.403+0000 7f9dc03e7640 1 -- 172.21.15.102:0/3357925092 wait complete. 2024-04-24T23:17:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dc03e7640 1 Processor -- start 2024-04-24T23:17:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dc03e7640 1 -- start start 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dc03e7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9db8101730 0x7f9db8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dc03e7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 0x7f9db81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dc03e7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db8107740 0x7f9db819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dc03e7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8111a00 con 0x7f9db81020f0 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dc03e7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8111880 con 0x7f9db8107740 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dc03e7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8111b80 con 0x7f9db8101730 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dbd982640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 0x7f9db81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dbd982640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 0x7f9db81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53186/0 (socket says 172.21.15.102:53186) 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dbd982640 1 -- 172.21.15.102:0/2153075739 learned_addr learned my addr 172.21.15.102:0/2153075739 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dbe183640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9db8101730 0x7f9db8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.404+0000 7f9dbe984640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db8107740 0x7f9db819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9dbd982640 1 -- 172.21.15.102:0/2153075739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9db8101730 msgr2=0x7f9db8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9dbd982640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9db8101730 0x7f9db8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9dbd982640 1 -- 172.21.15.102:0/2153075739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db8107740 msgr2=0x7f9db819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9dbd982640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db8107740 0x7f9db819c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9dbd982640 1 -- 172.21.15.102:0/2153075739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db819ce90 con 0x7f9db81020f0 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9dbd982640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 0x7f9db81983e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9da800d950 tx=0x7f9da800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9daf7fe640 1 -- 172.21.15.102:0/2153075739 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8017030 con 0x7f9db81020f0 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9daf7fe640 1 -- 172.21.15.102:0/2153075739 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9da8004020 con 0x7f9db81020f0 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9dc03e7640 1 -- 172.21.15.102:0/2153075739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db819d180 con 0x7f9db81020f0 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9daf7fe640 1 -- 172.21.15.102:0/2153075739 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da80047a0 con 0x7f9db81020f0 2024-04-24T23:17:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.406+0000 7f9dc03e7640 1 -- 172.21.15.102:0/2153075739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db81a4a10 con 0x7f9db81020f0 2024-04-24T23:17:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9daf7fe640 1 -- 172.21.15.102:0/2153075739 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f9da80041d0 con 0x7f9db81020f0 2024-04-24T23:17:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dc31d4740 1 Processor -- start 2024-04-24T23:17:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dc31d4740 1 -- start start 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26273620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273b60 0x55dd2627dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 0x55dd26280620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd261d7cb0 con 0x55dd26273240 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dd261805b0 con 0x55dd26273b60 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dd261d3c70 con 0x55dd2627e230 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26273620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26273620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53198/0 (socket says 172.21.15.102:53198) 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 -- 172.21.15.102:0/3531204555 learned_addr learned my addr 172.21.15.102:0/3531204555 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe183640 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273b60 0x55dd2627dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbd982640 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 0x55dd26280620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9daf7fe640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9da40a2e20 0x7f9da40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 -- 172.21.15.102:0/3531204555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 msgr2=0x55dd26280620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 0x55dd26280620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 -- 172.21.15.102:0/3531204555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273b60 msgr2=0x55dd2627dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273b60 0x55dd2627dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 -- 172.21.15.102:0/3531204555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd25eec210 con 0x55dd26273240 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbd982640 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 0x55dd26280620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9da40a2e20 0x7f9da40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9daf7fe640 1 -- 172.21.15.102:0/2153075739 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9da80cedb0 con 0x7f9db81020f0 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dbe984640 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26273620 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9db0031440 tx=0x7f9db0032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0ba611315cb9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dacbfe640 1 -- 172.21.15.102:0/3531204555 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0035040 con 0x55dd26273240 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dacbfe640 1 -- 172.21.15.102:0/3531204555 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9db0005510 con 0x55dd26273240 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.409+0000 7f9dacbfe640 1 -- 172.21.15.102:0/3531204555 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0046690 con 0x55dd26273240 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3531204555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 msgr2=0x55dd26273620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26273620 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9db0031440 tx=0x7f9db0032d00 comp rx=0 tx=0).stop 2024-04-24T23:17:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dbe984640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9da40a2e20 0x7f9da40a52e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55dd26280bf0 tx=0x7f9db0047000 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3531204555 shutdown_connections 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 0x55dd26280620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273b60 0x55dd2627dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/3531204555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26273620 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3531204555 >> 172.21.15.102:0/3531204555 conn(0x55dd2626f560 msgr2=0x55dd2626f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3531204555 shutdown_connections 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.410+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3531204555 wait complete. 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dc31d4740 1 -- start start 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dc31d4740 1 Processor -- start 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 0x55dd2627b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273b60 0x55dd2627b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 0x55dd2627bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd26281020 con 0x55dd26273b60 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dd26193b50 con 0x55dd26273240 2024-04-24T23:17:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dd262811a0 con 0x55dd2627e230 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dc31d4740 10 client.0 _ll_register_callbacks cb 0x55dd261ceca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbd982640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273b60 0x55dd2627b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbd982640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273b60 0x55dd2627b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53200/0 (socket says 172.21.15.102:53200) 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbd982640 1 -- 172.21.15.102:0/393556657 learned_addr learned my addr 172.21.15.102:0/393556657 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbe183640 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 0x55dd2627b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbd982640 1 -- 172.21.15.102:0/393556657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 msgr2=0x55dd2627bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbd982640 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 0x55dd2627bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbd982640 1 -- 172.21.15.102:0/393556657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 msgr2=0x55dd2627b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbd982640 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 0x55dd2627b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbd982640 1 -- 172.21.15.102:0/393556657 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd25fb8620 con 0x55dd26273b60 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9dbd982640 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273b60 0x55dd2627b690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9da80ccaf0 tx=0x7f9da809b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fce06e8bf17ad0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.411+0000 7f9da1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:17:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9da1ffb640 1 -- 172.21.15.102:0/393556657 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da809b2c0 con 0x55dd26273b60 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9da1ffb640 1 -- 172.21.15.102:0/393556657 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9da80bb0c0 con 0x55dd26273b60 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9dc31d4740 1 -- 172.21.15.102:0/393556657 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9db0005510 con 0x55dd26273b60 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9da1ffb640 1 -- 172.21.15.102:0/393556657 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9da809f670 con 0x55dd26273b60 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9dc31d4740 10 client.15753 fetch_fsmap learned FSMap version 26 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9dc31d4740 1 -- 172.21.15.102:0/393556657 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dd2628d1b0 con 0x55dd26273b60 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9da1ffb640 1 -- 172.21.15.102:0/393556657 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9db0005510 con 0x55dd26273b60 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9da1ffb640 1 -- 172.21.15.102:0/393556657 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9da809a710 con 0x55dd26273b60 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9dc31d4740 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9dc31d4740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9dc31d4740 20 client.15753 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55dd2628e620 0x55dd26290a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9dc31d4740 4 client.15753 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.412+0000 7f9dc31d4740 1 -- 172.21.15.102:0/393556657 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd261e51f0 con 0x55dd2628e620 2024-04-24T23:17:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.413+0000 7f9dbe183640 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55dd2628e620 0x55dd26290a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.413+0000 7f9dbe183640 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55dd2628e620 0x55dd26290a30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd2174d3ca781711 server_cookie=a7cea5de409f457d in_seq=0 out_seq=0 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.413+0000 7f9da1ffb640 10 client.15753 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9da1ffb640 1 -- 172.21.15.102:0/393556657 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd261e51f0 con 0x55dd2628e620 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9da1ffb640 10 client.15753 handle_command_reply: tid=0 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 client.15753 shutdown 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 -- 172.21.15.102:0/393556657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273b60 msgr2=0x55dd2627b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273b60 0x55dd2627b690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9da80ccaf0 tx=0x7f9da809b180 comp rx=0 tx=0).stop 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 -- 172.21.15.102:0/393556657 shutdown_connections 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd2627e230 0x55dd2627bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55dd2628e620 0x55dd26290a30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273b60 0x55dd2627b690 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/393556657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 0x55dd2627b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 -- 172.21.15.102:0/393556657 >> 172.21.15.102:0/393556657 conn(0x55dd2626f560 msgr2=0x55dd26275260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9da1ffb640 0 client.15753 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 -- 172.21.15.102:0/393556657 shutdown_connections 2024-04-24T23:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 1 -- 172.21.15.102:0/393556657 wait complete. 2024-04-24T23:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.414+0000 7f9dc31d4740 20 client.15753 trim_cache size 0 max 16384 2024-04-24T23:17:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.432+0000 7f9dc31d4740 1 Processor -- start 2024-04-24T23:17:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dc31d4740 1 -- start start 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 0x55dd26289ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 0x55dd2627a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd2627e230 0x55dd2626e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9db81afc70 con 0x55dd2627e230 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9db81ad540 con 0x55dd26273240 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9db81ab3b0 con 0x55dd26273b60 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe984640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd2627e230 0x55dd2626e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe984640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd2627e230 0x55dd2626e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53206/0 (socket says 172.21.15.102:53206) 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe984640 1 -- 172.21.15.102:0/3943764755 learned_addr learned my addr 172.21.15.102:0/3943764755 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe183640 1 --2- 172.21.15.102:0/3943764755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 0x55dd26289ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbd982640 1 --2- 172.21.15.102:0/3943764755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 0x55dd2627a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe984640 1 -- 172.21.15.102:0/3943764755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 msgr2=0x55dd2627a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe984640 1 --2- 172.21.15.102:0/3943764755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 0x55dd2627a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe984640 1 -- 172.21.15.102:0/3943764755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 msgr2=0x55dd26289ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe984640 1 --2- 172.21.15.102:0/3943764755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 0x55dd26289ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe984640 1 -- 172.21.15.102:0/3943764755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd26272850 con 0x55dd2627e230 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9dbe984640 1 --2- 172.21.15.102:0/3943764755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd2627e230 0x55dd2626e440 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9db002aab0 tx=0x7f9db0005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0c2883523fb412 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.433+0000 7f9da3fff640 1 -- 172.21.15.102:0/3943764755 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0035030 con 0x55dd2627e230 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9da3fff640 1 -- 172.21.15.102:0/3943764755 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9db0054070 con 0x55dd2627e230 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9da3fff640 1 -- 172.21.15.102:0/3943764755 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0046bc0 con 0x55dd2627e230 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3943764755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd2627e230 msgr2=0x55dd2626e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/3943764755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd2627e230 0x55dd2626e440 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9db002aab0 tx=0x7f9db0005d90 comp rx=0 tx=0).stop 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3943764755 shutdown_connections 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/3943764755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd2627e230 0x55dd2626e440 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/3943764755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 0x55dd2627a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/3943764755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd26273240 0x55dd26289ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3943764755 >> 172.21.15.102:0/3943764755 conn(0x55dd2626f560 msgr2=0x55dd2626dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3943764755 shutdown_connections 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.434+0000 7f9dc31d4740 1 -- 172.21.15.102:0/3943764755 wait complete. 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dc31d4740 1 -- start start 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dc31d4740 1 Processor -- start 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26286980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 0x55dd26277cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dc31d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd2627e230 0x55dd262783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9db81aba00 con 0x55dd26273240 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9db81ace70 con 0x55dd2627e230 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dc31d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9db81ac0d0 con 0x55dd26273b60 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dc31d4740 10 client.0 _ll_register_callbacks cb 0x7f9da8004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe183640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26286980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe183640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26286980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53220/0 (socket says 172.21.15.102:53220) 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe183640 1 -- 172.21.15.102:0/635499285 learned_addr learned my addr 172.21.15.102:0/635499285 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbd982640 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 0x55dd26277cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe984640 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd2627e230 0x55dd262783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe183640 1 -- 172.21.15.102:0/635499285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 msgr2=0x55dd26277cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe183640 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 0x55dd26277cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe183640 1 -- 172.21.15.102:0/635499285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd2627e230 msgr2=0x55dd262783f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe183640 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd2627e230 0x55dd262783f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe183640 1 -- 172.21.15.102:0/635499285 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd262f3bb0 con 0x55dd26273240 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe984640 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd2627e230 0x55dd262783f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbd982640 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 0x55dd26277cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9dbe183640 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26286980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55dd262912b0 tx=0x7f9db4015af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9d68e7e8305f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.435+0000 7f9da1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9da1ffb640 1 -- 172.21.15.102:0/635499285 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4003970 con 0x55dd26273240 2024-04-24T23:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9da1ffb640 1 -- 172.21.15.102:0/635499285 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f9db4003b10 con 0x55dd26273240 2024-04-24T23:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dc31d4740 1 -- 172.21.15.102:0/635499285 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9db0054070 con 0x55dd26273240 2024-04-24T23:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9da1ffb640 1 -- 172.21.15.102:0/635499285 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9db4003d10 con 0x55dd26273240 2024-04-24T23:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dc31d4740 10 client.15759 fetch_fsmap learned FSMap version 26 2024-04-24T23:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dc31d4740 1 -- 172.21.15.102:0/635499285 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dd262f4d60 con 0x55dd26273240 2024-04-24T23:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9da1ffb640 1 -- 172.21.15.102:0/635499285 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9db0054070 con 0x55dd26273240 2024-04-24T23:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9da1ffb640 1 -- 172.21.15.102:0/635499285 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9db400f040 con 0x55dd26273240 2024-04-24T23:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dc31d4740 10 client.15759 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dc31d4740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dc31d4740 20 client.15759 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55dd262f54b0 0x55dd262f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dc31d4740 4 client.15759 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dc31d4740 1 -- 172.21.15.102:0/635499285 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd261df540 con 0x55dd262f54b0 2024-04-24T23:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.436+0000 7f9dbd982640 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55dd262f54b0 0x55dd262f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.437+0000 7f9dbd982640 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55dd262f54b0 0x55dd262f78d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57525c35e2ffa9c7 server_cookie=e8f83d005efca4c0 in_seq=0 out_seq=0 2024-04-24T23:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.437+0000 7f9da1ffb640 10 client.15759 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9da1ffb640 1 -- 172.21.15.102:0/635499285 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd261df540 con 0x55dd262f54b0 2024-04-24T23:17:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9da1ffb640 10 client.15759 handle_command_reply: tid=0 2024-04-24T23:17:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 client.15759 shutdown 2024-04-24T23:17:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 -- 172.21.15.102:0/635499285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 msgr2=0x55dd26286980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26286980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55dd262912b0 tx=0x7f9db4015af0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 -- 172.21.15.102:0/635499285 shutdown_connections 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dd2627e230 0x55dd262783f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd26273b60 0x55dd26277cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55dd262f54b0 0x55dd262f78d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 --2- 172.21.15.102:0/635499285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd26273240 0x55dd26286980 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9da1ffb640 0 client.15759 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 -- 172.21.15.102:0/635499285 >> 172.21.15.102:0/635499285 conn(0x55dd2626f560 msgr2=0x55dd2626e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 -- 172.21.15.102:0/635499285 shutdown_connections 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 1 -- 172.21.15.102:0/635499285 wait complete. 2024-04-24T23:17:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.445+0000 7f9dc31d4740 20 client.15759 trim_cache size 0 max 16384 2024-04-24T23:17:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:17:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:17:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:17:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 -- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9da40a2e20 msgr2=0x7f9da40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9da40a2e20 0x7f9da40a52e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55dd26280bf0 tx=0x7f9db0047000 comp rx=0 tx=0).stop 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 -- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 msgr2=0x7f9db81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 0x7f9db81983e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9da800d950 tx=0x7f9da800de10 comp rx=0 tx=0).stop 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 -- 172.21.15.102:0/2153075739 shutdown_connections 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db8107740 0x7f9db819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9da40a2e20 0x7f9da40a52e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db81020f0 0x7f9db81983e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 --2- 172.21.15.102:0/2153075739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9db8101730 0x7f9db8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 -- 172.21.15.102:0/2153075739 >> 172.21.15.102:0/2153075739 conn(0x7f9db80fd200 msgr2=0x7f9db80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 -- 172.21.15.102:0/2153075739 shutdown_connections 2024-04-24T23:17:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:27.446+0000 7f9dc03e7640 1 -- 172.21.15.102:0/2153075739 wait complete. 2024-04-24T23:17:29.064 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:28 smithi183 ceph-mon[33137]: pgmap v136: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:17:29.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:28 smithi102 ceph-mon[29216]: pgmap v136: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:17:29.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:28 smithi161 ceph-mon[32709]: pgmap v136: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:17:31.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:30 smithi102 ceph-mon[29216]: pgmap v137: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:31.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:30 smithi161 ceph-mon[32709]: pgmap v137: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:31.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:30 smithi183 ceph-mon[33137]: pgmap v137: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:33.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:32 smithi102 ceph-mon[29216]: pgmap v138: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:33.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:32 smithi161 ceph-mon[32709]: pgmap v138: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:33.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:32 smithi183 ceph-mon[33137]: pgmap v138: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:35.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:34 smithi102 ceph-mon[29216]: pgmap v139: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:17:35.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:34 smithi161 ceph-mon[32709]: pgmap v139: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:17:35.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:34 smithi183 ceph-mon[33137]: pgmap v139: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 25 op/s 2024-04-24T23:17:37.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:36 smithi102 ceph-mon[29216]: pgmap v140: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:37.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:36 smithi161 ceph-mon[32709]: pgmap v140: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:37.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:36 smithi183 ceph-mon[33137]: pgmap v140: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:39.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:38 smithi102 ceph-mon[29216]: pgmap v141: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:39.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:38 smithi161 ceph-mon[32709]: pgmap v141: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:39.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:38 smithi183 ceph-mon[33137]: pgmap v141: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 -- 172.21.15.102:0/3018410057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 msgr2=0x7ff774100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 --2- 172.21.15.102:0/3018410057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 0x7ff774100950 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff764009900 tx=0x7ff76402a410 comp rx=0 tx=0).stop 2024-04-24T23:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 -- 172.21.15.102:0/3018410057 shutdown_connections 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 --2- 172.21.15.102:0/3018410057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff77410e120 0x7ff774110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 --2- 172.21.15.102:0/3018410057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff774100e90 0x7ff77410dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 --2- 172.21.15.102:0/3018410057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 0x7ff774100950 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 -- 172.21.15.102:0/3018410057 >> 172.21.15.102:0/3018410057 conn(0x7ff7740777c0 msgr2=0x7ff774077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 -- 172.21.15.102:0/3018410057 shutdown_connections 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 -- 172.21.15.102:0/3018410057 wait complete. 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.540+0000 7ff779716640 1 Processor -- start 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff779716640 1 -- start start 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff779716640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 0x7ff77419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff779716640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff774100e90 0x7ff77419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff779716640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff77410e120 0x7ff77419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff779716640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff774113be0 con 0x7ff774100570 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff779716640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff774113a60 con 0x7ff774100e90 2024-04-24T23:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff779716640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff774113d60 con 0x7ff77410e120 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff772ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 0x7ff77419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff772ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 0x7ff77419a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60710/0 (socket says 172.21.15.102:60710) 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff772ffd640 1 -- 172.21.15.102:0/1221810276 learned_addr learned my addr 172.21.15.102:0/1221810276 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff76bfff640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff774100e90 0x7ff77419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff7737fe640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff77410e120 0x7ff77419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff772ffd640 1 -- 172.21.15.102:0/1221810276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff77410e120 msgr2=0x7ff77419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff772ffd640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff77410e120 0x7ff77419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff772ffd640 1 -- 172.21.15.102:0/1221810276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff774100e90 msgr2=0x7ff77419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff772ffd640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff774100e90 0x7ff77419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.541+0000 7ff772ffd640 1 -- 172.21.15.102:0/1221810276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77419f0b0 con 0x7ff774100570 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.542+0000 7ff772ffd640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 0x7ff77419a0c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff764006010 tx=0x7ff764004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.542+0000 7ff770ff9640 1 -- 172.21.15.102:0/1221810276 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76403f030 con 0x7ff774100570 2024-04-24T23:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.542+0000 7ff770ff9640 1 -- 172.21.15.102:0/1221810276 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7640035f0 con 0x7ff774100570 2024-04-24T23:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.542+0000 7ff779716640 1 -- 172.21.15.102:0/1221810276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77419f340 con 0x7ff774100570 2024-04-24T23:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.542+0000 7ff770ff9640 1 -- 172.21.15.102:0/1221810276 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764035070 con 0x7ff774100570 2024-04-24T23:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.542+0000 7ff779716640 1 -- 172.21.15.102:0/1221810276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7741a6c30 con 0x7ff774100570 2024-04-24T23:17:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.544+0000 7ff770ff9640 1 -- 172.21.15.102:0/1221810276 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff764003b10 con 0x7ff774100570 2024-04-24T23:17:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.544+0000 7ff77c41a740 1 Processor -- start 2024-04-24T23:17:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.544+0000 7ff77c41a740 1 -- start start 2024-04-24T23:17:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cd240 0x55876d9cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cdb60 0x55876d9d7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 0x55876d9da620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff77c41a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55876d931cb0 con 0x55876d9cd240 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff77c41a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55876d8da5b0 con 0x55876d9cdb60 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff77c41a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55876d92dc70 con 0x55876d9d8230 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff7737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cd240 0x55876d9cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff7737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cd240 0x55876d9cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60712/0 (socket says 172.21.15.102:60712) 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff7737fe640 1 -- 172.21.15.102:0/3057977445 learned_addr learned my addr 172.21.15.102:0/3057977445 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff772ffd640 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cdb60 0x55876d9d7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff770ff9640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff7600a2da0 0x7ff7600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff76bfff640 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 0x55876d9da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff7737fe640 1 -- 172.21.15.102:0/3057977445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 msgr2=0x55876d9da620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff7737fe640 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 0x55876d9da620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff7737fe640 1 -- 172.21.15.102:0/3057977445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cdb60 msgr2=0x55876d9d7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff7737fe640 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cdb60 0x55876d9d7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff7737fe640 1 -- 172.21.15.102:0/3057977445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876d646210 con 0x55876d9cd240 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff772ffd640 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cdb60 0x55876d9d7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff770ff9640 1 -- 172.21.15.102:0/1221810276 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff7640efd20 con 0x7ff774100570 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.545+0000 7ff7737fe640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff7600a2da0 0x7ff7600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.546+0000 7ff7737fe640 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cd240 0x55876d9cd620 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff76c007f60 tx=0x7ff76c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c171c7e324304788 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.546+0000 7ff7697fa640 1 -- 172.21.15.102:0/3057977445 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c019870 con 0x55876d9cd240 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.546+0000 7ff7697fa640 1 -- 172.21.15.102:0/3057977445 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff76c010430 con 0x55876d9cd240 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.546+0000 7ff7697fa640 1 -- 172.21.15.102:0/3057977445 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c01a650 con 0x55876d9cd240 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 -- 172.21.15.102:0/3057977445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cd240 msgr2=0x55876d9cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff7737fe640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff7600a2da0 0x7ff7600a5260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff76c020000 tx=0x7ff76c01d000 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cd240 0x55876d9cd620 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff76c007f60 tx=0x7ff76c007f90 comp rx=0 tx=0).stop 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 -- 172.21.15.102:0/3057977445 shutdown_connections 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 0x55876d9da620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cdb60 0x55876d9d7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 --2- 172.21.15.102:0/3057977445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cd240 0x55876d9cd620 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 -- 172.21.15.102:0/3057977445 >> 172.21.15.102:0/3057977445 conn(0x55876d9c9560 msgr2=0x55876d9c9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 -- 172.21.15.102:0/3057977445 shutdown_connections 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 -- 172.21.15.102:0/3057977445 wait complete. 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 -- start start 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.547+0000 7ff77c41a740 1 Processor -- start 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 0x55876d9d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cdb60 0x55876d9d56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9d8230 0x55876d9d5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff77c41a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55876d9db020 con 0x55876d9d8230 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff77c41a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55876d8edb50 con 0x55876d9cd240 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff77c41a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55876d9db1a0 con 0x55876d9cdb60 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff77c41a740 10 client.0 _ll_register_callbacks cb 0x55876d928ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff7737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9d8230 0x55876d9d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff7737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9d8230 0x55876d9d5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60716/0 (socket says 172.21.15.102:60716) 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff7737fe640 1 -- 172.21.15.102:0/2229767237 learned_addr learned my addr 172.21.15.102:0/2229767237 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff76bfff640 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cdb60 0x55876d9d56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff772ffd640 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 0x55876d9d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff7737fe640 1 -- 172.21.15.102:0/2229767237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cdb60 msgr2=0x55876d9d56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff7737fe640 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cdb60 0x55876d9d56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff7737fe640 1 -- 172.21.15.102:0/2229767237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 msgr2=0x55876d9d5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff7737fe640 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 0x55876d9d5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff7737fe640 1 -- 172.21.15.102:0/2229767237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876d712620 con 0x55876d9d8230 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff7737fe640 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9d8230 0x55876d9d5c00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0108c0 tx=0x7ff76c0108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beeafceeef5e3b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.548+0000 7ff75a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff75a7fc640 1 -- 172.21.15.102:0/2229767237 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c01ab30 con 0x55876d9d8230 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff75a7fc640 1 -- 172.21.15.102:0/2229767237 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff76c01acd0 con 0x55876d9d8230 2024-04-24T23:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff77c41a740 1 -- 172.21.15.102:0/2229767237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff76c010430 con 0x55876d9d8230 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff75a7fc640 1 -- 172.21.15.102:0/2229767237 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff76c0197f0 con 0x55876d9d8230 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff77c41a740 10 client.15771 fetch_fsmap learned FSMap version 26 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff77c41a740 1 -- 172.21.15.102:0/2229767237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55876d9e6fd0 con 0x55876d9d8230 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff75a7fc640 1 -- 172.21.15.102:0/2229767237 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff76c010430 con 0x55876d9d8230 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff75a7fc640 1 -- 172.21.15.102:0/2229767237 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff76c02d070 con 0x55876d9d8230 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff77c41a740 10 client.15771 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff77c41a740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff77c41a740 20 client.15771 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff77c41a740 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55876d9e83e0 0x55876d9ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff77c41a740 4 client.15771 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.549+0000 7ff77c41a740 1 -- 172.21.15.102:0/2229767237 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55876d93f1f0 con 0x55876d9e83e0 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.550+0000 7ff772ffd640 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55876d9e83e0 0x55876d9ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.550+0000 7ff75a7fc640 10 client.15771 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.550+0000 7ff772ffd640 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55876d9e83e0 0x55876d9ea7d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2be82ff285713d server_cookie=52be3d58c463a233 in_seq=0 out_seq=0 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff75a7fc640 1 -- 172.21.15.102:0/2229767237 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55876d93f1f0 con 0x55876d9e83e0 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff75a7fc640 10 client.15771 handle_command_reply: tid=0 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 client.15771 shutdown 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 -- 172.21.15.102:0/2229767237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9d8230 msgr2=0x55876d9d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9d8230 0x55876d9d5c00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0108c0 tx=0x7ff76c0108f0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 -- 172.21.15.102:0/2229767237 shutdown_connections 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55876d9e83e0 0x55876d9ea7d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9d8230 0x55876d9d5c00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cdb60 0x55876d9d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff75a7fc640 0 client.15771 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 --2- 172.21.15.102:0/2229767237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 0x55876d9d5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 -- 172.21.15.102:0/2229767237 >> 172.21.15.102:0/2229767237 conn(0x55876d9c9560 msgr2=0x55876d9cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 -- 172.21.15.102:0/2229767237 shutdown_connections 2024-04-24T23:17:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 1 -- 172.21.15.102:0/2229767237 wait complete. 2024-04-24T23:17:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.551+0000 7ff77c41a740 20 client.15771 trim_cache size 0 max 16384 2024-04-24T23:17:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.571+0000 7ff77c41a740 1 Processor -- start 2024-04-24T23:17:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.571+0000 7ff77c41a740 1 -- start start 2024-04-24T23:17:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 0x55876d9d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 0x55876d9d5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff77c41a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741b1d60 con 0x55876d9cdb60 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff77c41a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741af630 con 0x55876d9cd240 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff77c41a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741ad4a0 con 0x55876d9d8230 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60730/0 (socket says 172.21.15.102:60730) 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff76bfff640 1 -- 172.21.15.102:0/4246135225 learned_addr learned my addr 172.21.15.102:0/4246135225 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff772ffd640 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 0x55876d9d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff7737fe640 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 0x55876d9d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff76bfff640 1 -- 172.21.15.102:0/4246135225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 msgr2=0x55876d9d5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff76bfff640 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 0x55876d9d5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff76bfff640 1 -- 172.21.15.102:0/4246135225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 msgr2=0x55876d9d78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff76bfff640 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 0x55876d9d78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff76bfff640 1 -- 172.21.15.102:0/4246135225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876d9d0dc0 con 0x55876d9cdb60 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff772ffd640 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 0x55876d9d78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.572+0000 7ff76bfff640 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9c8800 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00ab30 tx=0x7ff75c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bae345da529a868 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff768ff9640 1 -- 172.21.15.102:0/4246135225 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c020070 con 0x55876d9cdb60 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff768ff9640 1 -- 172.21.15.102:0/4246135225 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff75c012d90 con 0x55876d9cdb60 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff768ff9640 1 -- 172.21.15.102:0/4246135225 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c01b770 con 0x55876d9cdb60 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff77c41a740 1 -- 172.21.15.102:0/4246135225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 msgr2=0x55876d9c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff77c41a740 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9c8800 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00ab30 tx=0x7ff75c012450 comp rx=0 tx=0).stop 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff77c41a740 1 -- 172.21.15.102:0/4246135225 shutdown_connections 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff77c41a740 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9d8230 0x55876d9d5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff77c41a740 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9c8800 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff77c41a740 1 --2- 172.21.15.102:0/4246135225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9cd240 0x55876d9d78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.573+0000 7ff77c41a740 1 -- 172.21.15.102:0/4246135225 >> 172.21.15.102:0/4246135225 conn(0x55876d9c9560 msgr2=0x55876d9cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 -- 172.21.15.102:0/4246135225 shutdown_connections 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 -- 172.21.15.102:0/4246135225 wait complete. 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 -- start start 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 Processor -- start 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cd240 0x55876d9d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9d8230 0x55876d9d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741adaf0 con 0x55876d9cdb60 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741aef60 con 0x55876d9d8230 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741ae1c0 con 0x55876d9cd240 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff77c41a740 10 client.0 _ll_register_callbacks cb 0x7ff7640b8550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9d2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60734/0 (socket says 172.21.15.102:60734) 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff76bfff640 1 -- 172.21.15.102:0/1556844748 learned_addr learned my addr 172.21.15.102:0/1556844748 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff772ffd640 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cd240 0x55876d9d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.574+0000 7ff7737fe640 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9d8230 0x55876d9d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff76bfff640 1 -- 172.21.15.102:0/1556844748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cd240 msgr2=0x55876d9d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff76bfff640 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cd240 0x55876d9d2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff76bfff640 1 -- 172.21.15.102:0/1556844748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9d8230 msgr2=0x55876d9d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff76bfff640 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9d8230 0x55876d9d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff76bfff640 1 -- 172.21.15.102:0/1556844748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876da4da70 con 0x55876d9cdb60 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff772ffd640 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cd240 0x55876d9d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff76bfff640 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9d2690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff75c012f80 tx=0x7ff75c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c96833d48abcdd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff75a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff75a7fc640 1 -- 172.21.15.102:0/1556844748 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c020040 con 0x55876d9cdb60 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff75a7fc640 1 -- 172.21.15.102:0/1556844748 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff75c00f970 con 0x55876d9cdb60 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff77c41a740 1 -- 172.21.15.102:0/1556844748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff75c012d90 con 0x55876d9cdb60 2024-04-24T23:17:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff75a7fc640 1 -- 172.21.15.102:0/1556844748 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff75c033050 con 0x55876d9cdb60 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff77c41a740 10 client.15777 fetch_fsmap learned FSMap version 26 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff77c41a740 1 -- 172.21.15.102:0/1556844748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55876da4ed60 con 0x55876d9cdb60 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff75a7fc640 1 -- 172.21.15.102:0/1556844748 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff75c012d90 con 0x55876d9cdb60 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff75a7fc640 1 -- 172.21.15.102:0/1556844748 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff75c02e030 con 0x55876d9cdb60 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.575+0000 7ff77c41a740 10 client.15777 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.576+0000 7ff77c41a740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.576+0000 7ff77c41a740 20 client.15777 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.576+0000 7ff77c41a740 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55876da4f480 0x55876da518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.576+0000 7ff77c41a740 4 client.15777 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.576+0000 7ff77c41a740 1 -- 172.21.15.102:0/1556844748 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55876d939540 con 0x55876da4f480 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.576+0000 7ff772ffd640 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55876da4f480 0x55876da518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.576+0000 7ff75a7fc640 10 client.15777 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.576+0000 7ff772ffd640 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55876da4f480 0x55876da518a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82cabd3d8da9583 server_cookie=2f0d8fde0a8ae3bc in_seq=0 out_seq=0 2024-04-24T23:17:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff75a7fc640 1 -- 172.21.15.102:0/1556844748 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55876d939540 con 0x55876da4f480 2024-04-24T23:17:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff75a7fc640 10 client.15777 handle_command_reply: tid=0 2024-04-24T23:17:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 client.15777 shutdown 2024-04-24T23:17:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 -- 172.21.15.102:0/1556844748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 msgr2=0x55876d9d2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9d2690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff75c012f80 tx=0x7ff75c005750 comp rx=0 tx=0).stop 2024-04-24T23:17:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 -- 172.21.15.102:0/1556844748 shutdown_connections 2024-04-24T23:17:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55876d9d8230 0x55876d9d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55876da4f480 0x55876da518a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55876d9cdb60 0x55876d9d2690 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 --2- 172.21.15.102:0/1556844748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55876d9cd240 0x55876d9d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff75a7fc640 0 client.15777 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 -- 172.21.15.102:0/1556844748 >> 172.21.15.102:0/1556844748 conn(0x55876d9c9560 msgr2=0x55876d9e2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 -- 172.21.15.102:0/1556844748 shutdown_connections 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.583+0000 7ff77c41a740 1 -- 172.21.15.102:0/1556844748 wait complete. 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff77c41a740 20 client.15777 trim_cache size 0 max 16384 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:17:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:17:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:17:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 -- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff7600a2da0 msgr2=0x7ff7600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff7600a2da0 0x7ff7600a5260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff76c020000 tx=0x7ff76c01d000 comp rx=0 tx=0).stop 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 -- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 msgr2=0x7ff77419a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 0x7ff77419a0c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff764006010 tx=0x7ff764004270 comp rx=0 tx=0).stop 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 -- 172.21.15.102:0/1221810276 shutdown_connections 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff77410e120 0x7ff77419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff774100e90 0x7ff77419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff7600a2da0 0x7ff7600a5260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 --2- 172.21.15.102:0/1221810276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff774100570 0x7ff77419a0c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 -- 172.21.15.102:0/1221810276 >> 172.21.15.102:0/1221810276 conn(0x7ff7740777c0 msgr2=0x7ff77410f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 -- 172.21.15.102:0/1221810276 shutdown_connections 2024-04-24T23:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:40.584+0000 7ff779716640 1 -- 172.21.15.102:0/1221810276 wait complete. 2024-04-24T23:17:41.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:40 smithi102 ceph-mon[29216]: pgmap v142: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:41.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:40 smithi161 ceph-mon[32709]: pgmap v142: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:41.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:40 smithi183 ceph-mon[33137]: pgmap v142: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:43.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:42 smithi102 ceph-mon[29216]: pgmap v143: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:43.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:42 smithi161 ceph-mon[32709]: pgmap v143: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:43.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:42 smithi183 ceph-mon[33137]: pgmap v143: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:45.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:44 smithi102 ceph-mon[29216]: pgmap v144: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:17:45.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:44 smithi161 ceph-mon[32709]: pgmap v144: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:17:45.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:44 smithi183 ceph-mon[33137]: pgmap v144: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:17:47.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:46 smithi161 ceph-mon[32709]: pgmap v145: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:47.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:46 smithi183 ceph-mon[33137]: pgmap v145: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:47.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:46 smithi102 ceph-mon[29216]: pgmap v145: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:49.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:48 smithi161 ceph-mon[32709]: pgmap v146: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:49.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:48 smithi183 ceph-mon[33137]: pgmap v146: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:49.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:48 smithi102 ceph-mon[29216]: pgmap v146: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:50 smithi161 ceph-mon[32709]: pgmap v147: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:51.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:50 smithi183 ceph-mon[33137]: pgmap v147: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:51.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:50 smithi102 ceph-mon[29216]: pgmap v147: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:53.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:52 smithi161 ceph-mon[32709]: pgmap v148: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:53.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:52 smithi183 ceph-mon[33137]: pgmap v148: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:53.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:52 smithi102 ceph-mon[29216]: pgmap v148: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.682+0000 7fa99fa72640 1 -- 172.21.15.102:0/1488684346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998106630 msgr2=0x7fa998106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.682+0000 7fa99fa72640 1 --2- 172.21.15.102:0/1488684346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998106630 0x7fa998106a10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa994009930 tx=0x7fa99402a420 comp rx=0 tx=0).stop 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.682+0000 7fa99fa72640 1 -- 172.21.15.102:0/1488684346 shutdown_connections 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.682+0000 7fa99fa72640 1 --2- 172.21.15.102:0/1488684346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa998101030 0x7fa99810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.682+0000 7fa99fa72640 1 --2- 172.21.15.102:0/1488684346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa998100670 0x7fa998100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.682+0000 7fa99fa72640 1 --2- 172.21.15.102:0/1488684346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998106630 0x7fa998106a10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.682+0000 7fa99fa72640 1 -- 172.21.15.102:0/1488684346 >> 172.21.15.102:0/1488684346 conn(0x7fa9980777c0 msgr2=0x7fa998077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.682+0000 7fa99fa72640 1 -- 172.21.15.102:0/1488684346 shutdown_connections 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.682+0000 7fa99fa72640 1 -- 172.21.15.102:0/1488684346 wait complete. 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99fa72640 1 Processor -- start 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99fa72640 1 -- start start 2024-04-24T23:17:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99fa72640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa998100670 0x7fa998198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99fa72640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998101030 0x7fa9981992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99fa72640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa998106630 0x7fa9981a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99fa72640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981108b0 con 0x7fa998101030 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99fa72640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa998110730 con 0x7fa998106630 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99fa72640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa998110a30 con 0x7fa998100670 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99d00d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998101030 0x7fa9981992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99d00d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998101030 0x7fa9981992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60818/0 (socket says 172.21.15.102:60818) 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.683+0000 7fa99d00d640 1 -- 172.21.15.102:0/3597118140 learned_addr learned my addr 172.21.15.102:0/3597118140 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99d80e640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa998100670 0x7fa998198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99e00f640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa998106630 0x7fa9981a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99d00d640 1 -- 172.21.15.102:0/3597118140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa998100670 msgr2=0x7fa998198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99d00d640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa998100670 0x7fa998198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99d00d640 1 -- 172.21.15.102:0/3597118140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa998106630 msgr2=0x7fa9981a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99d00d640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa998106630 0x7fa9981a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99d00d640 1 -- 172.21.15.102:0/3597118140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9981a4350 con 0x7fa998101030 2024-04-24T23:17:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99d80e640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa998100670 0x7fa998198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99e00f640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa998106630 0x7fa9981a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99d00d640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998101030 0x7fa9981992d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa98800d950 tx=0x7fa98800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa98effd640 1 -- 172.21.15.102:0/3597118140 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988017030 con 0x7fa998101030 2024-04-24T23:17:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99fa72640 1 -- 172.21.15.102:0/3597118140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9981a4520 con 0x7fa998101030 2024-04-24T23:17:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa98effd640 1 -- 172.21.15.102:0/3597118140 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa988004020 con 0x7fa998101030 2024-04-24T23:17:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa98effd640 1 -- 172.21.15.102:0/3597118140 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9880047a0 con 0x7fa998101030 2024-04-24T23:17:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.684+0000 7fa99fa72640 1 -- 172.21.15.102:0/3597118140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9981a49e0 con 0x7fa998101030 2024-04-24T23:17:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.686+0000 7fa98effd640 1 -- 172.21.15.102:0/3597118140 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa9880041d0 con 0x7fa998101030 2024-04-24T23:17:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.687+0000 7fa9a277d740 1 Processor -- start 2024-04-24T23:17:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.687+0000 7fa9a277d740 1 -- start start 2024-04-24T23:17:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.687+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a23523a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.687+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 0x55a235249100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.687+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 0x55a23524ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.687+0000 7fa9a277d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a23519ecb0 con 0x55a235239f70 2024-04-24T23:17:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.687+0000 7fa9a277d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a2351475b0 con 0x55a235249640 2024-04-24T23:17:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.687+0000 7fa9a277d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a23519ac70 con 0x55a23523a920 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.687+0000 7fa98effd640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa9840a2da0 0x7fa9840a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a23523a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a23523a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60832/0 (socket says 172.21.15.102:60832) 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 -- 172.21.15.102:0/452773672 learned_addr learned my addr 172.21.15.102:0/452773672 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99d00d640 1 --2- 172.21.15.102:0/452773672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 0x55a23524ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99d80e640 1 --2- 172.21.15.102:0/452773672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 0x55a235249100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa9840a2da0 0x7fa9840a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa98effd640 1 -- 172.21.15.102:0/3597118140 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fa9880cefa0 con 0x7fa998101030 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 -- 172.21.15.102:0/452773672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 msgr2=0x55a235249100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 --2- 172.21.15.102:0/452773672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 0x55a235249100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 -- 172.21.15.102:0/452773672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 msgr2=0x55a23524ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 --2- 172.21.15.102:0/452773672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 0x55a23524ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 -- 172.21.15.102:0/452773672 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a234eb3210 con 0x55a235239f70 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 --2- 172.21.15.102:0/452773672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a23523a350 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa990014a10 tx=0x7fa990014ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b05ffb44197602d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa99e00f640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa9840a2da0 0x7fa9840a5260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa99001c000 tx=0x7fa99001e000 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.688+0000 7fa983fff640 1 -- 172.21.15.102:0/452773672 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990014050 con 0x55a235239f70 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa983fff640 1 -- 172.21.15.102:0/452773672 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa99000cd60 con 0x55a235239f70 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa983fff640 1 -- 172.21.15.102:0/452773672 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990010860 con 0x55a235239f70 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa9a277d740 1 -- 172.21.15.102:0/452773672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 msgr2=0x55a23523a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa9a277d740 1 --2- 172.21.15.102:0/452773672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a23523a350 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa990014a10 tx=0x7fa990014ed0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa9a277d740 1 -- 172.21.15.102:0/452773672 shutdown_connections 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa9a277d740 1 --2- 172.21.15.102:0/452773672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 0x55a23524ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa9a277d740 1 --2- 172.21.15.102:0/452773672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 0x55a235249100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa9a277d740 1 --2- 172.21.15.102:0/452773672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a23523a350 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa9a277d740 1 -- 172.21.15.102:0/452773672 >> 172.21.15.102:0/452773672 conn(0x55a235236560 msgr2=0x55a235236940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa9a277d740 1 -- 172.21.15.102:0/452773672 shutdown_connections 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.689+0000 7fa9a277d740 1 -- 172.21.15.102:0/452773672 wait complete. 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa9a277d740 1 -- start start 2024-04-24T23:17:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa9a277d740 1 Processor -- start 2024-04-24T23:17:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235246600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 0x55a235246b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 0x55a235247080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa9a277d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a23524c3a0 con 0x55a235239f70 2024-04-24T23:17:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa9a277d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a23515ab50 con 0x55a235249640 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa9a277d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a23524c520 con 0x55a23523a920 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa9a277d740 10 client.0 _ll_register_callbacks cb 0x55a235195ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa99d80e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235246600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.690+0000 7fa99d80e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235246600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60838/0 (socket says 172.21.15.102:60838) 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa99d80e640 1 -- 172.21.15.102:0/99154332 learned_addr learned my addr 172.21.15.102:0/99154332 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa99e00f640 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 0x55a235247080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa99d00d640 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 0x55a235246b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa99d80e640 1 -- 172.21.15.102:0/99154332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 msgr2=0x55a235246b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa99d80e640 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 0x55a235246b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa99d80e640 1 -- 172.21.15.102:0/99154332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 msgr2=0x55a235247080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa99d80e640 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 0x55a235247080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa99d80e640 1 -- 172.21.15.102:0/99154332 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a234f7f620 con 0x55a235239f70 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa99d80e640 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235246600 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa994033000 tx=0x7fa994031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f978db46f531834 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa9817fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:17:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa9817fa640 1 -- 172.21.15.102:0/99154332 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994035040 con 0x55a235239f70 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa9817fa640 1 -- 172.21.15.102:0/99154332 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fa994004460 con 0x55a235239f70 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa9a277d740 1 -- 172.21.15.102:0/99154332 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa99000cd60 con 0x55a235239f70 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa9817fa640 1 -- 172.21.15.102:0/99154332 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa9940318f0 con 0x55a235239f70 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa9a277d740 10 client.15789 fetch_fsmap learned FSMap version 26 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.691+0000 7fa9a277d740 1 -- 172.21.15.102:0/99154332 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2352585e0 con 0x55a235239f70 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.692+0000 7fa9817fa640 1 -- 172.21.15.102:0/99154332 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa99000cd60 con 0x55a235239f70 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.692+0000 7fa9817fa640 1 -- 172.21.15.102:0/99154332 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa994004cd0 con 0x55a235239f70 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.692+0000 7fa9a277d740 10 client.15789 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.692+0000 7fa9a277d740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.692+0000 7fa9a277d740 20 client.15789 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.692+0000 7fa9a277d740 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a235259aa0 0x55a23525beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.692+0000 7fa9a277d740 4 client.15789 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.692+0000 7fa9a277d740 1 -- 172.21.15.102:0/99154332 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2351ac1f0 con 0x55a235259aa0 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.692+0000 7fa99d00d640 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a235259aa0 0x55a23525beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9817fa640 10 client.15789 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa99d00d640 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a235259aa0 0x55a23525beb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92d75cfca704943 server_cookie=5b1f78d971f19f70 in_seq=0 out_seq=0 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9817fa640 1 -- 172.21.15.102:0/99154332 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2351ac1f0 con 0x55a235259aa0 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9817fa640 10 client.15789 handle_command_reply: tid=0 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9a277d740 1 client.15789 shutdown 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9a277d740 1 -- 172.21.15.102:0/99154332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 msgr2=0x55a235246600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9a277d740 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235246600 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa994033000 tx=0x7fa994031750 comp rx=0 tx=0).stop 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9a277d740 1 -- 172.21.15.102:0/99154332 shutdown_connections 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9a277d740 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a235249640 0x55a235247080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9a277d740 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a23523a920 0x55a235246b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9a277d740 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a235259aa0 0x55a23525beb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9a277d740 1 --2- 172.21.15.102:0/99154332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235246600 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9817fa640 0 client.15789 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.693+0000 7fa9a277d740 1 -- 172.21.15.102:0/99154332 >> 172.21.15.102:0/99154332 conn(0x55a235236560 msgr2=0x55a23523c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.694+0000 7fa9a277d740 1 -- 172.21.15.102:0/99154332 shutdown_connections 2024-04-24T23:17:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.694+0000 7fa9a277d740 1 -- 172.21.15.102:0/99154332 wait complete. 2024-04-24T23:17:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.694+0000 7fa9a277d740 20 client.15789 trim_cache size 0 max 16384 2024-04-24T23:17:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.708+0000 7fa9a277d740 1 Processor -- start 2024-04-24T23:17:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.708+0000 7fa9a277d740 1 -- start start 2024-04-24T23:17:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235255260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 0x55a235261810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 0x55a235261d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa9a277d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981ada80 con 0x55a235239f70 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa9a277d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981ab350 con 0x55a23523a920 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa9a277d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981a91c0 con 0x55a235249640 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d80e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235255260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d80e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235255260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60846/0 (socket says 172.21.15.102:60846) 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d80e640 1 -- 172.21.15.102:0/332911735 learned_addr learned my addr 172.21.15.102:0/332911735 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d00d640 1 --2- 172.21.15.102:0/332911735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 0x55a235261810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99e00f640 1 --2- 172.21.15.102:0/332911735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 0x55a235261d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d80e640 1 -- 172.21.15.102:0/332911735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 msgr2=0x55a235261d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d80e640 1 --2- 172.21.15.102:0/332911735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 0x55a235261d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d80e640 1 -- 172.21.15.102:0/332911735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 msgr2=0x55a235261810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d80e640 1 --2- 172.21.15.102:0/332911735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 0x55a235261810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d80e640 1 -- 172.21.15.102:0/332911735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a235244b50 con 0x55a235239f70 2024-04-24T23:17:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa99d80e640 1 --2- 172.21.15.102:0/332911735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235255260 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa9940035f0 tx=0x7fa994003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7740d3942503e8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.709+0000 7fa9837fe640 1 -- 172.21.15.102:0/332911735 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994050020 con 0x55a235239f70 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9837fe640 1 -- 172.21.15.102:0/332911735 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa994040070 con 0x55a235239f70 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9837fe640 1 -- 172.21.15.102:0/332911735 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940317c0 con 0x55a235239f70 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9a277d740 1 -- 172.21.15.102:0/332911735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 msgr2=0x55a235255260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9a277d740 1 --2- 172.21.15.102:0/332911735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235255260 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa9940035f0 tx=0x7fa994003f10 comp rx=0 tx=0).stop 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9a277d740 1 -- 172.21.15.102:0/332911735 shutdown_connections 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9a277d740 1 --2- 172.21.15.102:0/332911735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 0x55a235261d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9a277d740 1 --2- 172.21.15.102:0/332911735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 0x55a235261810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9a277d740 1 --2- 172.21.15.102:0/332911735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a235255260 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9a277d740 1 -- 172.21.15.102:0/332911735 >> 172.21.15.102:0/332911735 conn(0x55a235236560 msgr2=0x55a2352355c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9a277d740 1 -- 172.21.15.102:0/332911735 shutdown_connections 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.710+0000 7fa9a277d740 1 -- 172.21.15.102:0/332911735 wait complete. 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa9a277d740 1 -- start start 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa9a277d740 1 Processor -- start 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a2352517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 0x55a23524db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa9a277d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 0x55a23524e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa9a277d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981a9810 con 0x55a235239f70 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa9a277d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981aac80 con 0x55a23523a920 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa9a277d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981a9ee0 con 0x55a235249640 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa9a277d740 10 client.0 _ll_register_callbacks cb 0x7fa994009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa99d80e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a2352517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa99d80e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a2352517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60852/0 (socket says 172.21.15.102:60852) 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa99d80e640 1 -- 172.21.15.102:0/19662167 learned_addr learned my addr 172.21.15.102:0/19662167 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:17:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa99d00d640 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 0x55a23524db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.711+0000 7fa99e00f640 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 0x55a23524e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa99d80e640 1 -- 172.21.15.102:0/19662167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 msgr2=0x55a23524e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa99d80e640 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 0x55a23524e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa99d80e640 1 -- 172.21.15.102:0/19662167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 msgr2=0x55a23524db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa99d80e640 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 0x55a23524db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa99d80e640 1 -- 172.21.15.102:0/19662167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23524e5d0 con 0x55a235239f70 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa99d80e640 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a2352517d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa994006010 tx=0x7fa9940038e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327a751e41552d65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa9817fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa9817fa640 1 -- 172.21.15.102:0/19662167 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994050030 con 0x55a235239f70 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa9a277d740 1 -- 172.21.15.102:0/19662167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa994040070 con 0x55a235239f70 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa9817fa640 1 -- 172.21.15.102:0/19662167 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fa994005450 con 0x55a235239f70 2024-04-24T23:17:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa9817fa640 1 -- 172.21.15.102:0/19662167 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa994064050 con 0x55a235239f70 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa9a277d740 10 client.15795 fetch_fsmap learned FSMap version 26 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa9a277d740 1 -- 172.21.15.102:0/19662167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2352c0d50 con 0x55a235239f70 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.712+0000 7fa9817fa640 1 -- 172.21.15.102:0/19662167 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa994040070 con 0x55a235239f70 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa9817fa640 1 -- 172.21.15.102:0/19662167 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa9940048c0 con 0x55a235239f70 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa9a277d740 10 client.15795 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa9a277d740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa9a277d740 20 client.15795 populate_metadata read hostname 'smithi102' 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa9a277d740 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a2352c1350 0x55a2352c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa9a277d740 4 client.15795 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:17:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa9a277d740 1 -- 172.21.15.102:0/19662167 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2351a6540 con 0x55a2352c1350 2024-04-24T23:17:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa99d00d640 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a2352c1350 0x55a2352c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa9817fa640 10 client.15795 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.713+0000 7fa99d00d640 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a2352c1350 0x55a2352c3740 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2fed88cc6a40c4 server_cookie=25efe4dab974a0df in_seq=0 out_seq=0 2024-04-24T23:17:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9817fa640 1 -- 172.21.15.102:0/19662167 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2351a6540 con 0x55a2352c1350 2024-04-24T23:17:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9817fa640 10 client.15795 handle_command_reply: tid=0 2024-04-24T23:17:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 client.15795 shutdown 2024-04-24T23:17:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 -- 172.21.15.102:0/19662167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 msgr2=0x55a2352517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a2352517d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa994006010 tx=0x7fa9940038e0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 -- 172.21.15.102:0/19662167 shutdown_connections 2024-04-24T23:17:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a235249640 0x55a23524e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a23523a920 0x55a23524db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a2352c1350 0x55a2352c3740 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 --2- 172.21.15.102:0/19662167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a235239f70 0x55a2352517d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9817fa640 0 client.15795 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:17:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 -- 172.21.15.102:0/19662167 >> 172.21.15.102:0/19662167 conn(0x55a235236560 msgr2=0x55a23523bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 -- 172.21.15.102:0/19662167 shutdown_connections 2024-04-24T23:17:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 1 -- 172.21.15.102:0/19662167 wait complete. 2024-04-24T23:17:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.721+0000 7fa9a277d740 20 client.15795 trim_cache size 0 max 16384 2024-04-24T23:17:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:17:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:17:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:17:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 -- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa9840a2da0 msgr2=0x7fa9840a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa9840a2da0 0x7fa9840a5260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa99001c000 tx=0x7fa99001e000 comp rx=0 tx=0).stop 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 -- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998101030 msgr2=0x7fa9981992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998101030 0x7fa9981992d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa98800d950 tx=0x7fa98800de10 comp rx=0 tx=0).stop 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 -- 172.21.15.102:0/3597118140 shutdown_connections 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa998106630 0x7fa9981a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa9840a2da0 0x7fa9840a5260 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa998101030 0x7fa9981992d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 --2- 172.21.15.102:0/3597118140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa998100670 0x7fa998198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 -- 172.21.15.102:0/3597118140 >> 172.21.15.102:0/3597118140 conn(0x7fa9980777c0 msgr2=0x7fa998103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 -- 172.21.15.102:0/3597118140 shutdown_connections 2024-04-24T23:17:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:17:53.722+0000 7fa99fa72640 1 -- 172.21.15.102:0/3597118140 wait complete. 2024-04-24T23:17:55.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:54 smithi161 ceph-mon[32709]: pgmap v149: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:17:55.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:54 smithi183 ceph-mon[33137]: pgmap v149: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:17:55.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:54 smithi102 ceph-mon[29216]: pgmap v149: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:17:57.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:56 smithi161 ceph-mon[32709]: pgmap v150: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:57.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:56 smithi183 ceph-mon[33137]: pgmap v150: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:56 smithi102 ceph-mon[29216]: pgmap v150: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:17:59.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:17:58 smithi161 ceph-mon[32709]: pgmap v151: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:59.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:17:58 smithi183 ceph-mon[33137]: pgmap v151: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:17:59.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:17:58 smithi102 ceph-mon[29216]: pgmap v151: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:01.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:00 smithi161 ceph-mon[32709]: pgmap v152: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:01.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:00 smithi183 ceph-mon[33137]: pgmap v152: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:01.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:00 smithi102 ceph-mon[29216]: pgmap v152: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:03.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:02 smithi161 ceph-mon[32709]: pgmap v153: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:03.123 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:02 smithi102 ceph-mon[29216]: pgmap v153: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:03.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:02 smithi183 ceph-mon[33137]: pgmap v153: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:05.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:04 smithi161 ceph-mon[32709]: pgmap v154: 97 pgs: 97 active+clean; 6.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:04 smithi183 ceph-mon[33137]: pgmap v154: 97 pgs: 97 active+clean; 6.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:18:05.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:04 smithi102 ceph-mon[29216]: pgmap v154: 97 pgs: 97 active+clean; 6.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.821+0000 7f583e02a640 1 -- 172.21.15.102:0/2192956790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 msgr2=0x7f5838100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.821+0000 7f583e02a640 1 --2- 172.21.15.102:0/2192956790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 0x7f5838100b10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f582c009930 tx=0x7f582c02a420 comp rx=0 tx=0).stop 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.821+0000 7f583e02a640 1 -- 172.21.15.102:0/2192956790 shutdown_connections 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.821+0000 7f583e02a640 1 --2- 172.21.15.102:0/2192956790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5838101050 0x7f583810d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.821+0000 7f583e02a640 1 --2- 172.21.15.102:0/2192956790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 0x7f5838100b10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.821+0000 7f583e02a640 1 --2- 172.21.15.102:0/2192956790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5838106650 0x7f5838106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.821+0000 7f583e02a640 1 -- 172.21.15.102:0/2192956790 >> 172.21.15.102:0/2192956790 conn(0x7f5838077790 msgr2=0x7f5838077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.821+0000 7f583e02a640 1 -- 172.21.15.102:0/2192956790 shutdown_connections 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.821+0000 7f583e02a640 1 -- 172.21.15.102:0/2192956790 wait complete. 2024-04-24T23:18:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f583e02a640 1 Processor -- start 2024-04-24T23:18:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f583e02a640 1 -- start start 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f583e02a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 0x7f5838197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f583e02a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5838101050 0x7f5838198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f583e02a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5838106650 0x7f583819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f583e02a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5838110910 con 0x7f5838100690 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f583e02a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5838110790 con 0x7f5838106650 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f583e02a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5838110a90 con 0x7f5838101050 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f58377fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 0x7f5838197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f58377fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 0x7f5838197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47650/0 (socket says 172.21.15.102:47650) 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.822+0000 7f58377fe640 1 -- 172.21.15.102:0/1677292018 learned_addr learned my addr 172.21.15.102:0/1677292018 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f5836ffd640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5838101050 0x7f5838198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f5837fff640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5838106650 0x7f583819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f58377fe640 1 -- 172.21.15.102:0/1677292018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5838101050 msgr2=0x7f5838198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f58377fe640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5838101050 0x7f5838198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f58377fe640 1 -- 172.21.15.102:0/1677292018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5838106650 msgr2=0x7f583819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f58377fe640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5838106650 0x7f583819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f58377fe640 1 -- 172.21.15.102:0/1677292018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583819cf30 con 0x7f5838100690 2024-04-24T23:18:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f58377fe640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 0x7f5838197f40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f582800b4a0 tx=0x7f582800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f5834ff9640 1 -- 172.21.15.102:0/1677292018 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828014070 con 0x7f5838100690 2024-04-24T23:18:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f5834ff9640 1 -- 172.21.15.102:0/1677292018 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5828010ac0 con 0x7f5838100690 2024-04-24T23:18:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f583e02a640 1 -- 172.21.15.102:0/1677292018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583819d220 con 0x7f5838100690 2024-04-24T23:18:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f5834ff9640 1 -- 172.21.15.102:0/1677292018 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828017c30 con 0x7f5838100690 2024-04-24T23:18:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.823+0000 7f583e02a640 1 -- 172.21.15.102:0/1677292018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58381a4ab0 con 0x7f5838100690 2024-04-24T23:18:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5834ff9640 1 -- 172.21.15.102:0/1677292018 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f5828017070 con 0x7f5838100690 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5840e12740 1 Processor -- start 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5840e12740 1 -- start start 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5834ff9640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f58200a2e20 0x7f58200a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5840e12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a0596620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5840e12740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 0x5592a05a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5840e12740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a05a3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5840e12740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5592a04facb0 con 0x5592a05a1230 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5840e12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5592a04a35b0 con 0x5592a0596240 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.826+0000 7f5840e12740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5592a04f6c70 con 0x5592a0596b60 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f5837fff640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f58200a2e20 0x7f58200a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f58377fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a0596620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f5837fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a05a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f58377fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a0596620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:58772/0 (socket says 172.21.15.102:58772) 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f58377fe640 1 -- 172.21.15.102:0/65048735 learned_addr learned my addr 172.21.15.102:0/65048735 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f5836ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 0x5592a05a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f5834ff9640 1 -- 172.21.15.102:0/1677292018 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f58280ce5e0 con 0x7f5838100690 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f58377fe640 1 -- 172.21.15.102:0/65048735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 msgr2=0x5592a05a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f58377fe640 1 --2- 172.21.15.102:0/65048735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 0x5592a05a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f58377fe640 1 -- 172.21.15.102:0/65048735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 msgr2=0x5592a05a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f58377fe640 1 --2- 172.21.15.102:0/65048735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a05a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f58377fe640 1 -- 172.21.15.102:0/65048735 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a020f210 con 0x5592a0596240 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f5837fff640 1 --2- 172.21.15.102:0/65048735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a05a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f5837fff640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f58200a2e20 0x7f58200a52e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58300098b0 tx=0x7f5830009e20 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f58377fe640 1 --2- 172.21.15.102:0/65048735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a0596620 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5828009420 tx=0x7f58280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efda2215f482e486 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.827+0000 7f581dffb640 1 -- 172.21.15.102:0/65048735 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582801e070 con 0x5592a0596240 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f581dffb640 1 -- 172.21.15.102:0/65048735 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582809acb0 con 0x5592a0596240 2024-04-24T23:18:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f581dffb640 1 -- 172.21.15.102:0/65048735 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280a1550 con 0x5592a0596240 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f5840e12740 1 -- 172.21.15.102:0/65048735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 msgr2=0x5592a0596620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f5840e12740 1 --2- 172.21.15.102:0/65048735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a0596620 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5828009420 tx=0x7f58280046c0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f5840e12740 1 -- 172.21.15.102:0/65048735 shutdown_connections 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f5840e12740 1 --2- 172.21.15.102:0/65048735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a05a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f5840e12740 1 --2- 172.21.15.102:0/65048735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 0x5592a05a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f5840e12740 1 --2- 172.21.15.102:0/65048735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a0596620 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f5840e12740 1 -- 172.21.15.102:0/65048735 >> 172.21.15.102:0/65048735 conn(0x5592a0592560 msgr2=0x5592a0592940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f5840e12740 1 -- 172.21.15.102:0/65048735 shutdown_connections 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.828+0000 7f5840e12740 1 -- 172.21.15.102:0/65048735 wait complete. 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5840e12740 1 -- start start 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5840e12740 1 Processor -- start 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5840e12740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596240 0x5592a059e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5840e12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596b60 0x5592a059e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5840e12740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a059ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5840e12740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5592a05a4020 con 0x5592a05a1230 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5840e12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5592a04b6b50 con 0x5592a0596b60 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5840e12740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5592a05a41a0 con 0x5592a0596240 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5840e12740 10 client.0 _ll_register_callbacks cb 0x5592a04f1ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5837fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a059ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5836ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596b60 0x5592a059e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5837fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a059ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47662/0 (socket says 172.21.15.102:47662) 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5836ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596b60 0x5592a059e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:58788/0 (socket says 172.21.15.102:58788) 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f5837fff640 1 -- 172.21.15.102:0/1969526401 learned_addr learned my addr 172.21.15.102:0/1969526401 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.829+0000 7f58377fe640 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596240 0x5592a059e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5837fff640 1 -- 172.21.15.102:0/1969526401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596240 msgr2=0x5592a059e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5837fff640 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596240 0x5592a059e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5837fff640 1 -- 172.21.15.102:0/1969526401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596b60 msgr2=0x5592a059e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5837fff640 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596b60 0x5592a059e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5837fff640 1 -- 172.21.15.102:0/1969526401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a02db620 con 0x5592a05a1230 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5836ffd640 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596b60 0x5592a059e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5837fff640 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a059ebd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5830010ea0 tx=0x7f5830009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fa0f401a188ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5816ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5816ffd640 1 -- 172.21.15.102:0/1969526401 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58300045b0 con 0x5592a05a1230 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5816ffd640 1 -- 172.21.15.102:0/1969526401 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5830004750 con 0x5592a05a1230 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5840e12740 1 -- 172.21.15.102:0/1969526401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582809acb0 con 0x5592a05a1230 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5816ffd640 1 -- 172.21.15.102:0/1969526401 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5830029480 con 0x5592a05a1230 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5840e12740 10 client.15807 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5840e12740 1 -- 172.21.15.102:0/1969526401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592a05b01b0 con 0x5592a05a1230 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.830+0000 7f5816ffd640 1 -- 172.21.15.102:0/1969526401 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582809acb0 con 0x5592a05a1230 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f5816ffd640 1 -- 172.21.15.102:0/1969526401 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f583001f070 con 0x5592a05a1230 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f5840e12740 10 client.15807 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f5840e12740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f5840e12740 20 client.15807 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f5840e12740 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5592a05b1620 0x5592a05b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f5840e12740 4 client.15807 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f5840e12740 1 -- 172.21.15.102:0/1969526401 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592a05081f0 con 0x5592a05b1620 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f58377fe640 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5592a05b1620 0x5592a05b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f58377fe640 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5592a05b1620 0x5592a05b3a10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669fa01f5915d778 server_cookie=9baf53e3389e1580 in_seq=0 out_seq=0 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.831+0000 7f5816ffd640 10 client.15807 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5816ffd640 1 -- 172.21.15.102:0/1969526401 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592a05081f0 con 0x5592a05b1620 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5816ffd640 10 client.15807 handle_command_reply: tid=0 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 client.15807 shutdown 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 -- 172.21.15.102:0/1969526401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 msgr2=0x5592a059ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a059ebd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5830010ea0 tx=0x7f5830009600 comp rx=0 tx=0).stop 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 -- 172.21.15.102:0/1969526401 shutdown_connections 2024-04-24T23:18:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5592a05b1620 0x5592a05b3a10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a05a1230 0x5592a059ebd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596b60 0x5592a059e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 --2- 172.21.15.102:0/1969526401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596240 0x5592a059e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5816ffd640 0 client.15807 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 -- 172.21.15.102:0/1969526401 >> 172.21.15.102:0/1969526401 conn(0x5592a0592560 msgr2=0x5592a0598360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 -- 172.21.15.102:0/1969526401 shutdown_connections 2024-04-24T23:18:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.832+0000 7f5840e12740 1 -- 172.21.15.102:0/1969526401 wait complete. 2024-04-24T23:18:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.833+0000 7f5840e12740 20 client.15807 trim_cache size 0 max 16384 2024-04-24T23:18:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.850+0000 7f5840e12740 1 Processor -- start 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.850+0000 7f5840e12740 1 -- start start 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.850+0000 7f5840e12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a05a0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.850+0000 7f5840e12740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596b60 0x5592a05acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5840e12740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a05a1230 0x5592a059d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5840e12740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58381afd60 con 0x5592a0596b60 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5840e12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f58381ad630 con 0x5592a0596240 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5840e12740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58381ab4a0 con 0x5592a05a1230 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5836ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596b60 0x5592a05acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5836ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596b60 0x5592a05acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47668/0 (socket says 172.21.15.102:47668) 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f58377fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a05a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5836ffd640 1 -- 172.21.15.102:0/522216468 learned_addr learned my addr 172.21.15.102:0/522216468 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5837fff640 1 --2- 172.21.15.102:0/522216468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a05a1230 0x5592a059d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5836ffd640 1 -- 172.21.15.102:0/522216468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a05a1230 msgr2=0x5592a059d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5836ffd640 1 --2- 172.21.15.102:0/522216468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a05a1230 0x5592a059d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5836ffd640 1 -- 172.21.15.102:0/522216468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 msgr2=0x5592a05a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5836ffd640 1 --2- 172.21.15.102:0/522216468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a05a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5836ffd640 1 -- 172.21.15.102:0/522216468 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a05b3d80 con 0x5592a0596b60 2024-04-24T23:18:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f5836ffd640 1 --2- 172.21.15.102:0/522216468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596b60 0x5592a05acee0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f582c009930 tx=0x7f582c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8719a9cb332eec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f581d7fa640 1 -- 172.21.15.102:0/522216468 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c035040 con 0x5592a0596b60 2024-04-24T23:18:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f581d7fa640 1 -- 172.21.15.102:0/522216468 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c031860 con 0x5592a0596b60 2024-04-24T23:18:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.851+0000 7f581d7fa640 1 -- 172.21.15.102:0/522216468 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c004a00 con 0x5592a0596b60 2024-04-24T23:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.852+0000 7f5840e12740 1 -- 172.21.15.102:0/522216468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596b60 msgr2=0x5592a05acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.852+0000 7f5840e12740 1 --2- 172.21.15.102:0/522216468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596b60 0x5592a05acee0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f582c009930 tx=0x7f582c005fd0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.852+0000 7f5840e12740 1 -- 172.21.15.102:0/522216468 shutdown_connections 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.852+0000 7f5840e12740 1 --2- 172.21.15.102:0/522216468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a05a1230 0x5592a059d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.852+0000 7f5840e12740 1 --2- 172.21.15.102:0/522216468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596b60 0x5592a05acee0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.852+0000 7f5840e12740 1 --2- 172.21.15.102:0/522216468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a0596240 0x5592a05a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.852+0000 7f5840e12740 1 -- 172.21.15.102:0/522216468 >> 172.21.15.102:0/522216468 conn(0x5592a0592560 msgr2=0x5592a0592940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.852+0000 7f5840e12740 1 -- 172.21.15.102:0/522216468 shutdown_connections 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.852+0000 7f5840e12740 1 -- 172.21.15.102:0/522216468 wait complete. 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5840e12740 1 -- start start 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5840e12740 1 Processor -- start 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5840e12740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596240 0x5592a05a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5840e12740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 0x5592a05a9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5840e12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a05a1230 0x5592a059a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5840e12740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58381abaf0 con 0x5592a0596240 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5840e12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f58381acf60 con 0x5592a05a1230 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5840e12740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58381ac1c0 con 0x5592a0596b60 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5840e12740 10 client.0 _ll_register_callbacks cb 0x7f5830010b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f58377fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596240 0x5592a05a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f58377fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596240 0x5592a05a93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47684/0 (socket says 172.21.15.102:47684) 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f58377fe640 1 -- 172.21.15.102:0/2201300302 learned_addr learned my addr 172.21.15.102:0/2201300302 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5836ffd640 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 0x5592a05a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f5837fff640 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a05a1230 0x5592a059a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.853+0000 7f58377fe640 1 -- 172.21.15.102:0/2201300302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 msgr2=0x5592a05a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f58377fe640 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 0x5592a05a9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f58377fe640 1 -- 172.21.15.102:0/2201300302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a05a1230 msgr2=0x5592a059a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f58377fe640 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a05a1230 0x5592a059a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f58377fe640 1 -- 172.21.15.102:0/2201300302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a059ab20 con 0x5592a0596240 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5837fff640 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a05a1230 0x5592a059a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f58377fe640 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596240 0x5592a05a93f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5828009420 tx=0x7f582800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899730f3a3c31990 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5816ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5816ffd640 1 -- 172.21.15.102:0/2201300302 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582801e040 con 0x5592a0596240 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5816ffd640 1 -- 172.21.15.102:0/2201300302 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5828098fb0 con 0x5592a0596240 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5840e12740 1 -- 172.21.15.102:0/2201300302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582c031860 con 0x5592a0596240 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5816ffd640 1 -- 172.21.15.102:0/2201300302 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f58280a3870 con 0x5592a0596240 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5840e12740 10 client.15813 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5840e12740 1 -- 172.21.15.102:0/2201300302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592a0617500 con 0x5592a0596240 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5816ffd640 1 -- 172.21.15.102:0/2201300302 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c031860 con 0x5592a0596240 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5816ffd640 1 -- 172.21.15.102:0/2201300302 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f58280a0800 con 0x5592a0596240 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5840e12740 10 client.15813 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5840e12740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5840e12740 20 client.15813 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5840e12740 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5592a0617800 0x5592a0619bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5840e12740 4 client.15813 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:18:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.854+0000 7f5840e12740 1 -- 172.21.15.102:0/2201300302 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592a0502540 con 0x5592a0617800 2024-04-24T23:18:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.855+0000 7f5836ffd640 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5592a0617800 0x5592a0619bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.855+0000 7f5816ffd640 10 client.15813 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.855+0000 7f5836ffd640 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5592a0617800 0x5592a0619bf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb55aedd9f4d45e server_cookie=d3c1389c15d59aca in_seq=0 out_seq=0 2024-04-24T23:18:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5816ffd640 1 -- 172.21.15.102:0/2201300302 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592a0502540 con 0x5592a0617800 2024-04-24T23:18:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5816ffd640 10 client.15813 handle_command_reply: tid=0 2024-04-24T23:18:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 client.15813 shutdown 2024-04-24T23:18:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 -- 172.21.15.102:0/2201300302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596240 msgr2=0x5592a05a93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596240 0x5592a05a93f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5828009420 tx=0x7f582800bf90 comp rx=0 tx=0).stop 2024-04-24T23:18:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 -- 172.21.15.102:0/2201300302 shutdown_connections 2024-04-24T23:18:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592a05a1230 0x5592a059a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592a0596b60 0x5592a05a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5592a0617800 0x5592a0619bf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 --2- 172.21.15.102:0/2201300302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5592a0596240 0x5592a05a93f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 -- 172.21.15.102:0/2201300302 >> 172.21.15.102:0/2201300302 conn(0x5592a0592560 msgr2=0x5592a059e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5816ffd640 0 client.15813 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 -- 172.21.15.102:0/2201300302 shutdown_connections 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 1 -- 172.21.15.102:0/2201300302 wait complete. 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.864+0000 7f5840e12740 20 client.15813 trim_cache size 0 max 16384 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:18:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:18:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:18:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 -- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f58200a2e20 msgr2=0x7f58200a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f58200a2e20 0x7f58200a52e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58300098b0 tx=0x7f5830009e20 comp rx=0 tx=0).stop 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 -- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 msgr2=0x7f5838197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 0x7f5838197f40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f582800b4a0 tx=0x7f582800b960 comp rx=0 tx=0).stop 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 -- 172.21.15.102:0/1677292018 shutdown_connections 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5838106650 0x7f583819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5838101050 0x7f5838198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f58200a2e20 0x7f58200a52e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 --2- 172.21.15.102:0/1677292018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5838100690 0x7f5838197f40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 -- 172.21.15.102:0/1677292018 >> 172.21.15.102:0/1677292018 conn(0x7f5838077790 msgr2=0x7f5838069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 -- 172.21.15.102:0/1677292018 shutdown_connections 2024-04-24T23:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:06.865+0000 7f583e02a640 1 -- 172.21.15.102:0/1677292018 wait complete. 2024-04-24T23:18:07.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:06 smithi161 ceph-mon[32709]: pgmap v155: 97 pgs: 97 active+clean; 6.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:18:07.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:06 smithi183 ceph-mon[33137]: pgmap v155: 97 pgs: 97 active+clean; 6.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:18:07.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:06 smithi102 ceph-mon[29216]: pgmap v155: 97 pgs: 97 active+clean; 6.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:18:09.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:08 smithi161 ceph-mon[32709]: pgmap v156: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 41 op/s 2024-04-24T23:18:09.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:08 smithi183 ceph-mon[33137]: pgmap v156: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 41 op/s 2024-04-24T23:18:09.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:08 smithi102 ceph-mon[29216]: pgmap v156: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 41 op/s 2024-04-24T23:18:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:10 smithi161 ceph-mon[32709]: pgmap v157: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:18:11.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:10 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:10 smithi183 ceph-mon[33137]: pgmap v157: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:18:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:10 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:11.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:10 smithi102 ceph-mon[29216]: pgmap v157: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:18:11.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:10 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:12.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:11 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:12.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:11 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:12.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:11 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:18:12.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:11 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:12.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:11 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:12.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:11 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:18:12.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:11 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:12.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:11 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:12.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:11 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:18:13.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:12 smithi161 ceph-mon[32709]: pgmap v158: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:18:13.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:12 smithi183 ceph-mon[33137]: pgmap v158: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:18:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:12 smithi102 ceph-mon[29216]: pgmap v158: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:18:15.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:14 smithi161 ceph-mon[32709]: pgmap v159: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:18:15.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:14 smithi183 ceph-mon[33137]: pgmap v159: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:18:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:14 smithi102 ceph-mon[29216]: pgmap v159: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:18:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:16 smithi183 ceph-mon[33137]: pgmap v160: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:17.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:16 smithi102 ceph-mon[29216]: pgmap v160: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:17.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:16 smithi161 ceph-mon[32709]: pgmap v160: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:19.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:18 smithi183 ceph-mon[33137]: pgmap v161: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:19.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:18 smithi102 ceph-mon[29216]: pgmap v161: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:18 smithi161 ceph-mon[32709]: pgmap v161: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.978+0000 7faaebbfb640 1 -- 172.21.15.102:0/2374798452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 msgr2=0x7faae4100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.978+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2374798452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 0x7faae4100af0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faad4009930 tx=0x7faad402a420 comp rx=0 tx=0).stop 2024-04-24T23:18:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.978+0000 7faaebbfb640 1 -- 172.21.15.102:0/2374798452 shutdown_connections 2024-04-24T23:18:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.978+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2374798452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae4101030 0x7faae410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.978+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2374798452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 0x7faae4100af0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.978+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2374798452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4106630 0x7faae4106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.978+0000 7faaebbfb640 1 -- 172.21.15.102:0/2374798452 >> 172.21.15.102:0/2374798452 conn(0x7faae40777c0 msgr2=0x7faae4077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.978+0000 7faaebbfb640 1 -- 172.21.15.102:0/2374798452 shutdown_connections 2024-04-24T23:18:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.978+0000 7faaebbfb640 1 -- 172.21.15.102:0/2374798452 wait complete. 2024-04-24T23:18:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faaebbfb640 1 Processor -- start 2024-04-24T23:18:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faaebbfb640 1 -- start start 2024-04-24T23:18:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faaebbfb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 0x7faae419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faaebbfb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae4101030 0x7faae419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faaebbfb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4106630 0x7faae419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faaebbfb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faae41119f0 con 0x7faae4100670 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faaebbfb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faae4111870 con 0x7faae4106630 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faaebbfb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faae4111b70 con 0x7faae4101030 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faae9997640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 0x7faae419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faae9997640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 0x7faae419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58506/0 (socket says 172.21.15.102:58506) 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faae9997640 1 -- 172.21.15.102:0/2443076877 learned_addr learned my addr 172.21.15.102:0/2443076877 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.979+0000 7faae9196640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae4101030 0x7faae419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faaea198640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4106630 0x7faae419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faae9997640 1 -- 172.21.15.102:0/2443076877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae4101030 msgr2=0x7faae419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faae9997640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae4101030 0x7faae419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faae9997640 1 -- 172.21.15.102:0/2443076877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4106630 msgr2=0x7faae419ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faae9997640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4106630 0x7faae419ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faae9997640 1 -- 172.21.15.102:0/2443076877 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae419f140 con 0x7faae4100670 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faae9997640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 0x7faae419a150 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faae000e9f0 tx=0x7faae000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faadaffd640 1 -- 172.21.15.102:0/2443076877 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae000cd80 con 0x7faae4100670 2024-04-24T23:18:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faaebbfb640 1 -- 172.21.15.102:0/2443076877 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae419f430 con 0x7faae4100670 2024-04-24T23:18:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faadaffd640 1 -- 172.21.15.102:0/2443076877 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faae00108e0 con 0x7faae4100670 2024-04-24T23:18:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faadaffd640 1 -- 172.21.15.102:0/2443076877 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0018920 con 0x7faae4100670 2024-04-24T23:18:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.980+0000 7faaebbfb640 1 -- 172.21.15.102:0/2443076877 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae41a6cc0 con 0x7faae4100670 2024-04-24T23:18:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.982+0000 7faadaffd640 1 -- 172.21.15.102:0/2443076877 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7faae0010a90 con 0x7faae4100670 2024-04-24T23:18:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.983+0000 7faaee9e0740 1 Processor -- start 2024-04-24T23:18:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.983+0000 7faaee9e0740 1 -- start start 2024-04-24T23:18:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.983+0000 7faadaffd640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faad00a2da0 0x7faad00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.983+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x5651251f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x565125202620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faaee9e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x565125159cb0 con 0x5651251f5240 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faaee9e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651251025b0 con 0x565125200230 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faaee9e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565125155c70 con 0x5651251f5b60 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faaea198640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faad00a2da0 0x7faad00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faadaffd640 1 -- 172.21.15.102:0/2443076877 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7faae00cf7a0 con 0x7faae4100670 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9997640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x5651251f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9997640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x5651251f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58522/0 (socket says 172.21.15.102:58522) 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9997640 1 -- 172.21.15.102:0/2424562728 learned_addr learned my addr 172.21.15.102:0/2424562728 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faaea198640 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x565125202620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9196640 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9997640 1 -- 172.21.15.102:0/2424562728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 msgr2=0x5651251ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9997640 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9997640 1 -- 172.21.15.102:0/2424562728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 msgr2=0x565125202620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9997640 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x565125202620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9997640 1 -- 172.21.15.102:0/2424562728 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565124e6e210 con 0x5651251f5240 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9196640 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faaea198640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faad00a2da0 0x7faad00a5260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faadc005fd0 tx=0x7faadc014ab0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faae9997640 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x5651251f5620 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faae0009380 tx=0x7faae007fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40de3a0977f0aabc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faacffff640 1 -- 172.21.15.102:0/2424562728 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae009c3b0 con 0x5651251f5240 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faacffff640 1 -- 172.21.15.102:0/2424562728 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faae00833b0 con 0x5651251f5240 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.984+0000 7faacffff640 1 -- 172.21.15.102:0/2424562728 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae007fb70 con 0x5651251f5240 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.985+0000 7faaee9e0740 1 -- 172.21.15.102:0/2424562728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 msgr2=0x5651251f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.985+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x5651251f5620 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faae0009380 tx=0x7faae007fa50 comp rx=0 tx=0).stop 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.985+0000 7faaee9e0740 1 -- 172.21.15.102:0/2424562728 shutdown_connections 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.985+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x565125202620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.985+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.985+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2424562728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x5651251f5620 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.985+0000 7faaee9e0740 1 -- 172.21.15.102:0/2424562728 >> 172.21.15.102:0/2424562728 conn(0x5651251f1560 msgr2=0x5651251f1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.985+0000 7faaee9e0740 1 -- 172.21.15.102:0/2424562728 shutdown_connections 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.985+0000 7faaee9e0740 1 -- 172.21.15.102:0/2424562728 wait complete. 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaee9e0740 1 -- start start 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaee9e0740 1 Processor -- start 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5240 0x5651251fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5b60 0x5651251fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x5651251fdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaee9e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x565125202fc0 con 0x5651251f5b60 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaee9e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565125115b50 con 0x565125200230 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaee9e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565125203140 con 0x5651251f5240 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaee9e0740 10 client.0 _ll_register_callbacks cb 0x565125150ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faae9196640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5b60 0x5651251fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faae9196640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5b60 0x5651251fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58534/0 (socket says 172.21.15.102:58534) 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faae9196640 1 -- 172.21.15.102:0/12225073 learned_addr learned my addr 172.21.15.102:0/12225073 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faae9997640 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5240 0x5651251fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.986+0000 7faaea198640 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x5651251fdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faae9196640 1 -- 172.21.15.102:0/12225073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5240 msgr2=0x5651251fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faae9196640 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5240 0x5651251fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faae9196640 1 -- 172.21.15.102:0/12225073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 msgr2=0x5651251fdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faae9196640 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x5651251fdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faae9196640 1 -- 172.21.15.102:0/12225073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565124f3a620 con 0x5651251f5b60 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faae9196640 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5b60 0x5651251fd720 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faad4009930 tx=0x7faad4031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb634968794eb3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faacd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faacd7fa640 1 -- 172.21.15.102:0/12225073 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4035040 con 0x5651251f5b60 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faacd7fa640 1 -- 172.21.15.102:0/12225073 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7faad40316a0 con 0x5651251f5b60 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faaee9e0740 1 -- 172.21.15.102:0/12225073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae00833b0 con 0x5651251f5b60 2024-04-24T23:18:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faacd7fa640 1 -- 172.21.15.102:0/12225073 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faad40318a0 con 0x5651251f5b60 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faaee9e0740 10 client.15825 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faaee9e0740 1 -- 172.21.15.102:0/12225073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56512520f1e0 con 0x5651251f5b60 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.987+0000 7faacd7fa640 1 -- 172.21.15.102:0/12225073 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faae00833b0 con 0x5651251f5b60 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faacd7fa640 1 -- 172.21.15.102:0/12225073 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faad4004830 con 0x5651251f5b60 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faaee9e0740 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faaee9e0740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faaee9e0740 20 client.15825 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faaee9e0740 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565125210760 0x565125212b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faaee9e0740 4 client.15825 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faaee9e0740 1 -- 172.21.15.102:0/12225073 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651251671f0 con 0x565125210760 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faae9997640 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565125210760 0x565125212b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faae9997640 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565125210760 0x565125212b50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6685b86a1041744 server_cookie=8becf32032ea0252 in_seq=0 out_seq=0 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.988+0000 7faacd7fa640 10 client.15825 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.989+0000 7faacd7fa640 1 -- 172.21.15.102:0/12225073 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651251671f0 con 0x565125210760 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.989+0000 7faacd7fa640 10 client.15825 handle_command_reply: tid=0 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.989+0000 7faaee9e0740 1 client.15825 shutdown 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 -- 172.21.15.102:0/12225073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5b60 msgr2=0x5651251fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5b60 0x5651251fd720 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faad4009930 tx=0x7faad4031e60 comp rx=0 tx=0).stop 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 -- 172.21.15.102:0/12225073 shutdown_connections 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x5651251fdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565125210760 0x565125212b50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5b60 0x5651251fd720 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 --2- 172.21.15.102:0/12225073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5240 0x5651251fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faacd7fa640 0 client.15825 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 -- 172.21.15.102:0/12225073 >> 172.21.15.102:0/12225073 conn(0x5651251f1560 msgr2=0x5651251f7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 -- 172.21.15.102:0/12225073 shutdown_connections 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 1 -- 172.21.15.102:0/12225073 wait complete. 2024-04-24T23:18:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:19.990+0000 7faaee9e0740 20 client.15825 trim_cache size 0 max 16384 2024-04-24T23:18:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faaee9e0740 1 Processor -- start 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faaee9e0740 1 -- start start 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x56512520be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x565125218420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faaee9e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faae41b1ec0 con 0x5651251f5240 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faaee9e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faae41af790 con 0x565125200230 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faaee9e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faae41ad600 con 0x5651251f5b60 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faae9997640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x56512520be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.006+0000 7faae9997640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x56512520be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58536/0 (socket says 172.21.15.102:58536) 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faae9997640 1 -- 172.21.15.102:0/1003771407 learned_addr learned my addr 172.21.15.102:0/1003771407 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faaea198640 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x565125218420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faae9196640 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faae9997640 1 -- 172.21.15.102:0/1003771407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 msgr2=0x5651251f79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faae9997640 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251f79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faae9997640 1 -- 172.21.15.102:0/1003771407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 msgr2=0x565125218420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faae9997640 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x565125218420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faae9997640 1 -- 172.21.15.102:0/1003771407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651252131b0 con 0x5651251f5240 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faae9196640 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251f79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faae9997640 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x56512520be80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faae009b9b0 tx=0x7faae0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa45c8540db2b722 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faacf7fe640 1 -- 172.21.15.102:0/1003771407 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae009c3b0 con 0x5651251f5240 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faacf7fe640 1 -- 172.21.15.102:0/1003771407 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faae00a4920 con 0x5651251f5240 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.007+0000 7faacf7fe640 1 -- 172.21.15.102:0/1003771407 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae007fd20 con 0x5651251f5240 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 -- 172.21.15.102:0/1003771407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 msgr2=0x56512520be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x56512520be80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faae009b9b0 tx=0x7faae0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 -- 172.21.15.102:0/1003771407 shutdown_connections 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565125200230 0x565125218420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651251f5b60 0x5651251f79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 --2- 172.21.15.102:0/1003771407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x56512520be80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 -- 172.21.15.102:0/1003771407 >> 172.21.15.102:0/1003771407 conn(0x5651251f1560 msgr2=0x5651251f8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 -- 172.21.15.102:0/1003771407 shutdown_connections 2024-04-24T23:18:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 -- 172.21.15.102:0/1003771407 wait complete. 2024-04-24T23:18:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 -- start start 2024-04-24T23:18:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.008+0000 7faaee9e0740 1 Processor -- start 2024-04-24T23:18:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x565125208380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651251f5b60 0x5651252088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faaee9e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565125200230 0x565125203af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faaee9e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faae41adc50 con 0x5651251f5240 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faaee9e0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faae41af0c0 con 0x5651251f5b60 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faaee9e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faae41ae320 con 0x565125200230 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faaee9e0740 10 client.0 _ll_register_callbacks cb 0x7faad4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9997640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x565125208380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9997640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x565125208380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58538/0 (socket says 172.21.15.102:58538) 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9997640 1 -- 172.21.15.102:0/2041752565 learned_addr learned my addr 172.21.15.102:0/2041752565 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9196640 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651251f5b60 0x5651252088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9997640 1 -- 172.21.15.102:0/2041752565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565125200230 msgr2=0x565125203af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faaea198640 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565125200230 0x565125203af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9997640 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565125200230 0x565125203af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9997640 1 -- 172.21.15.102:0/2041752565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651251f5b60 msgr2=0x5651252088c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9997640 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651251f5b60 0x5651252088c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9997640 1 -- 172.21.15.102:0/2041752565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651252042f0 con 0x5651251f5240 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faae9997640 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x565125208380 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7faae00a43f0 tx=0x7faae00a0060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60667c644f708b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.009+0000 7faacd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faacd7fa640 1 -- 172.21.15.102:0/2041752565 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae009c3b0 con 0x5651251f5240 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faacd7fa640 1 -- 172.21.15.102:0/2041752565 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7faae0098320 con 0x5651251f5240 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faaee9e0740 1 -- 172.21.15.102:0/2041752565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae00a4920 con 0x5651251f5240 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faacd7fa640 1 -- 172.21.15.102:0/2041752565 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faae00a4b70 con 0x5651251f5240 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faaee9e0740 10 client.15831 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faaee9e0740 1 -- 172.21.15.102:0/2041752565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651252768a0 con 0x5651251f5240 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faacd7fa640 1 -- 172.21.15.102:0/2041752565 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faae00a4920 con 0x5651251f5240 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faacd7fa640 1 -- 172.21.15.102:0/2041752565 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faae009b100 con 0x5651251f5240 2024-04-24T23:18:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faaee9e0740 10 client.15831 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faaee9e0740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faaee9e0740 20 client.15831 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565125276bc0 0x565125278fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faaee9e0740 4 client.15831 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:18:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faaee9e0740 1 -- 172.21.15.102:0/2041752565 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565125161540 con 0x565125276bc0 2024-04-24T23:18:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.010+0000 7faae9196640 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565125276bc0 0x565125278fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.011+0000 7faae9196640 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565125276bc0 0x565125278fb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=872182f035b2ff87 server_cookie=32799300cd5eb85d in_seq=0 out_seq=0 2024-04-24T23:18:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.011+0000 7faacd7fa640 10 client.15831 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faacd7fa640 1 -- 172.21.15.102:0/2041752565 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565125161540 con 0x565125276bc0 2024-04-24T23:18:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faacd7fa640 10 client.15831 handle_command_reply: tid=0 2024-04-24T23:18:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 client.15831 shutdown 2024-04-24T23:18:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 -- 172.21.15.102:0/2041752565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 msgr2=0x565125208380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x565125208380 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7faae00a43f0 tx=0x7faae00a0060 comp rx=0 tx=0).stop 2024-04-24T23:18:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 -- 172.21.15.102:0/2041752565 shutdown_connections 2024-04-24T23:18:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565125200230 0x565125203af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651251f5b60 0x5651252088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x565125276bc0 0x565125278fb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 --2- 172.21.15.102:0/2041752565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651251f5240 0x565125208380 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faacd7fa640 0 client.15831 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 -- 172.21.15.102:0/2041752565 >> 172.21.15.102:0/2041752565 conn(0x5651251f1560 msgr2=0x5651251efd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.018+0000 7faaee9e0740 1 -- 172.21.15.102:0/2041752565 shutdown_connections 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaee9e0740 1 -- 172.21.15.102:0/2041752565 wait complete. 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaee9e0740 20 client.15831 trim_cache size 0 max 16384 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:18:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:18:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 -- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faad00a2da0 msgr2=0x7faad00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faad00a2da0 0x7faad00a5260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faadc005fd0 tx=0x7faadc014ab0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 -- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 msgr2=0x7faae419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 0x7faae419a150 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faae000e9f0 tx=0x7faae000eeb0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 -- 172.21.15.102:0/2443076877 shutdown_connections 2024-04-24T23:18:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faae4106630 0x7faae419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae4101030 0x7faae419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7faad00a2da0 0x7faad00a5260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 --2- 172.21.15.102:0/2443076877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4100670 0x7faae419a150 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 -- 172.21.15.102:0/2443076877 >> 172.21.15.102:0/2443076877 conn(0x7faae40777c0 msgr2=0x7faae410b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 -- 172.21.15.102:0/2443076877 shutdown_connections 2024-04-24T23:18:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:20.019+0000 7faaebbfb640 1 -- 172.21.15.102:0/2443076877 wait complete. 2024-04-24T23:18:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:20 smithi183 ceph-mon[33137]: pgmap v162: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-04-24T23:18:21.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:20 smithi102 ceph-mon[29216]: pgmap v162: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-04-24T23:18:21.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:20 smithi161 ceph-mon[32709]: pgmap v162: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-04-24T23:18:23.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:22 smithi183 ceph-mon[33137]: pgmap v163: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-04-24T23:18:23.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:22 smithi102 ceph-mon[29216]: pgmap v163: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-04-24T23:18:23.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:22 smithi161 ceph-mon[32709]: pgmap v163: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 15 op/s 2024-04-24T23:18:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:24 smithi183 ceph-mon[33137]: pgmap v164: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:25.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:24 smithi102 ceph-mon[29216]: pgmap v164: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:25.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:24 smithi161 ceph-mon[32709]: pgmap v164: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:27.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:26 smithi183 ceph-mon[33137]: pgmap v165: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:18:27.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:26 smithi102 ceph-mon[29216]: pgmap v165: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:18:27.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:26 smithi161 ceph-mon[32709]: pgmap v165: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:18:29.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:28 smithi183 ceph-mon[33137]: pgmap v166: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:29.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:28 smithi102 ceph-mon[29216]: pgmap v166: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:29.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:28 smithi161 ceph-mon[32709]: pgmap v166: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:31.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:30 smithi183 ceph-mon[33137]: pgmap v167: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:31.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:30 smithi102 ceph-mon[29216]: pgmap v167: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:31.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:30 smithi161 ceph-mon[32709]: pgmap v167: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.116+0000 7f85fe190640 1 -- 172.21.15.102:0/2891650608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8106630 msgr2=0x7f85f8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.116+0000 7f85fe190640 1 --2- 172.21.15.102:0/2891650608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8106630 0x7f85f8106a10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009930 tx=0x7f85e802a420 comp rx=0 tx=0).stop 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.116+0000 7f85fe190640 1 -- 172.21.15.102:0/2891650608 shutdown_connections 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.116+0000 7f85fe190640 1 --2- 172.21.15.102:0/2891650608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85f8101030 0x7f85f810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.116+0000 7f85fe190640 1 --2- 172.21.15.102:0/2891650608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85f8100670 0x7f85f8100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.116+0000 7f85fe190640 1 --2- 172.21.15.102:0/2891650608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8106630 0x7f85f8106a10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.116+0000 7f85fe190640 1 -- 172.21.15.102:0/2891650608 >> 172.21.15.102:0/2891650608 conn(0x7f85f80777c0 msgr2=0x7f85f8077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.116+0000 7f85fe190640 1 -- 172.21.15.102:0/2891650608 shutdown_connections 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.116+0000 7f85fe190640 1 -- 172.21.15.102:0/2891650608 wait complete. 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85fe190640 1 Processor -- start 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85fe190640 1 -- start start 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85fe190640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8100670 0x7f85f80fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85fe190640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85f8101030 0x7f85f80fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85fe190640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85f8106630 0x7f85f80fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85fe190640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81108b0 con 0x7f85f8100670 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85fe190640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85f8110730 con 0x7f85f8106630 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85fe190640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85f8110a30 con 0x7f85f8101030 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8100670 0x7f85f80fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8100670 0x7f85f80fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56332/0 (socket says 172.21.15.102:56332) 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f77fe640 1 -- 172.21.15.102:0/2145816972 learned_addr learned my addr 172.21.15.102:0/2145816972 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f6ffd640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85f8101030 0x7f85f80fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f7fff640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85f8106630 0x7f85f80fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f77fe640 1 -- 172.21.15.102:0/2145816972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85f8101030 msgr2=0x7f85f80fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f77fe640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85f8101030 0x7f85f80fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f77fe640 1 -- 172.21.15.102:0/2145816972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85f8106630 msgr2=0x7f85f80fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f77fe640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85f8106630 0x7f85f80fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f77fe640 1 -- 172.21.15.102:0/2145816972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f81a6350 con 0x7f85f8100670 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.117+0000 7f85f7fff640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85f8106630 0x7f85f80fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:18:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.118+0000 7f85f77fe640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8100670 0x7f85f80fd600 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f85e8004770 tx=0x7f85e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.118+0000 7f85f4ff9640 1 -- 172.21.15.102:0/2145816972 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e803f070 con 0x7f85f8100670 2024-04-24T23:18:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.118+0000 7f85f4ff9640 1 -- 172.21.15.102:0/2145816972 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85e803b460 con 0x7f85f8100670 2024-04-24T23:18:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.118+0000 7f85fe190640 1 -- 172.21.15.102:0/2145816972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f81a65e0 con 0x7f85f8100670 2024-04-24T23:18:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.118+0000 7f85f4ff9640 1 -- 172.21.15.102:0/2145816972 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8035070 con 0x7f85f8100670 2024-04-24T23:18:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.118+0000 7f85fe190640 1 -- 172.21.15.102:0/2145816972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f81a6ad0 con 0x7f85f8100670 2024-04-24T23:18:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.120+0000 7f85f4ff9640 1 -- 172.21.15.102:0/2145816972 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f85e803b610 con 0x7f85f8100670 2024-04-24T23:18:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.120+0000 7f8600f7d740 1 Processor -- start 2024-04-24T23:18:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.120+0000 7f8600f7d740 1 -- start start 2024-04-24T23:18:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 0x563c913e8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913e8b60 0x563c913f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913f3230 0x563c913f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f8600f7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563c9134ccb0 con 0x563c913e8b60 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f8600f7d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563c912f55b0 con 0x563c913e8240 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f8600f7d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563c91348c70 con 0x563c913f3230 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f4ff9640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85e00a2da0 0x7f85e00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913e8b60 0x563c913f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 0x563c913e8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 0x563c913e8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:54782/0 (socket says 172.21.15.102:54782) 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f7fff640 1 -- 172.21.15.102:0/1415552231 learned_addr learned my addr 172.21.15.102:0/1415552231 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f6ffd640 1 --2- 172.21.15.102:0/1415552231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913f3230 0x563c913f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f7fff640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85e00a2da0 0x7f85e00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f4ff9640 1 -- 172.21.15.102:0/2145816972 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f85e80ef7e0 con 0x7f85f8100670 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f6ffd640 1 -- 172.21.15.102:0/1415552231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 msgr2=0x563c913e8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f6ffd640 1 --2- 172.21.15.102:0/1415552231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 0x563c913e8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f6ffd640 1 -- 172.21.15.102:0/1415552231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913e8b60 msgr2=0x563c913f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f6ffd640 1 --2- 172.21.15.102:0/1415552231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913e8b60 0x563c913f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.121+0000 7f85f6ffd640 1 -- 172.21.15.102:0/1415552231 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c91061210 con 0x563c913f3230 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.122+0000 7f85f77fe640 1 --2- 172.21.15.102:0/1415552231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913e8b60 0x563c913f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.122+0000 7f85f7fff640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85e00a2da0 0x7f85e00a5260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85f0005c40 tx=0x7f85f0009a80 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.122+0000 7f85f6ffd640 1 --2- 172.21.15.102:0/1415552231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913f3230 0x563c913f5620 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00abc0 tx=0x7f85ec00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56df35d9553e868 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.122+0000 7f85ddffb640 1 -- 172.21.15.102:0/1415552231 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec005220 con 0x563c913f3230 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.122+0000 7f85ddffb640 1 -- 172.21.15.102:0/1415552231 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85ec0047a0 con 0x563c913f3230 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.122+0000 7f85ddffb640 1 -- 172.21.15.102:0/1415552231 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0186a0 con 0x563c913f3230 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 -- 172.21.15.102:0/1415552231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913f3230 msgr2=0x563c913f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 --2- 172.21.15.102:0/1415552231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913f3230 0x563c913f5620 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00abc0 tx=0x7f85ec00f790 comp rx=0 tx=0).stop 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 -- 172.21.15.102:0/1415552231 shutdown_connections 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 --2- 172.21.15.102:0/1415552231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913f3230 0x563c913f5620 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 --2- 172.21.15.102:0/1415552231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913e8b60 0x563c913f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 --2- 172.21.15.102:0/1415552231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 0x563c913e8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 -- 172.21.15.102:0/1415552231 >> 172.21.15.102:0/1415552231 conn(0x563c913e4560 msgr2=0x563c913e4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 -- 172.21.15.102:0/1415552231 shutdown_connections 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 -- 172.21.15.102:0/1415552231 wait complete. 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 -- start start 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.123+0000 7f8600f7d740 1 Processor -- start 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 0x563c913f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8b60 0x563c913f05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f8600f7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563c913f6020 con 0x563c913f3230 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f8600f7d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563c91308b50 con 0x563c913e8240 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f8600f7d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563c913f61a0 con 0x563c913e8b60 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f8600f7d740 10 client.0 _ll_register_callbacks cb 0x563c91343ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56346/0 (socket says 172.21.15.102:56346) 2024-04-24T23:18:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f7fff640 1 -- 172.21.15.102:0/594841289 learned_addr learned my addr 172.21.15.102:0/594841289 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f77fe640 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 0x563c913f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f6ffd640 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8b60 0x563c913f05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f7fff640 1 -- 172.21.15.102:0/594841289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8b60 msgr2=0x563c913f05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f7fff640 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8b60 0x563c913f05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f7fff640 1 -- 172.21.15.102:0/594841289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 msgr2=0x563c913f00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f7fff640 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 0x563c913f00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f7fff640 1 -- 172.21.15.102:0/594841289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c9112d620 con 0x563c913f3230 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85f7fff640 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f0b30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f85f0012eb0 tx=0x7f85f0009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ca582c239d4cda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.124+0000 7f85d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f85d6ffd640 1 -- 172.21.15.102:0/594841289 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0004b20 con 0x563c913f3230 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f85d6ffd640 1 -- 172.21.15.102:0/594841289 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f85f0004cc0 con 0x563c913f3230 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f8600f7d740 1 -- 172.21.15.102:0/594841289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ec0047a0 con 0x563c913f3230 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f85d6ffd640 1 -- 172.21.15.102:0/594841289 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85f0011af0 con 0x563c913f3230 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f8600f7d740 10 client.15840 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f8600f7d740 1 -- 172.21.15.102:0/594841289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c91402080 con 0x563c913f3230 2024-04-24T23:18:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f85d6ffd640 1 -- 172.21.15.102:0/594841289 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f0025070 con 0x563c913f3230 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f85d6ffd640 1 -- 172.21.15.102:0/594841289 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f85f001f070 con 0x563c913f3230 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f8600f7d740 10 client.15840 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f8600f7d740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f8600f7d740 20 client.15840 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f8600f7d740 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563c91403720 0x563c91405b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f8600f7d740 4 client.15840 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.125+0000 7f8600f7d740 1 -- 172.21.15.102:0/594841289 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f85ec0047a0 con 0x563c91403720 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.126+0000 7f85f77fe640 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563c91403720 0x563c91405b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.126+0000 7f85f77fe640 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563c91403720 0x563c91405b10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf871a98bfeafca server_cookie=b46490e79b3b1fa5 in_seq=0 out_seq=0 2024-04-24T23:18:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.126+0000 7f85d6ffd640 10 client.15840 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f85d6ffd640 1 -- 172.21.15.102:0/594841289 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f85ec0047a0 con 0x563c91403720 2024-04-24T23:18:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f85d6ffd640 10 client.15840 handle_command_reply: tid=0 2024-04-24T23:18:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 client.15840 shutdown 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 -- 172.21.15.102:0/594841289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 msgr2=0x563c913f0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f0b30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f85f0012eb0 tx=0x7f85f0009760 comp rx=0 tx=0).stop 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 -- 172.21.15.102:0/594841289 shutdown_connections 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563c91403720 0x563c91405b10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f0b30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8b60 0x563c913f05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 --2- 172.21.15.102:0/594841289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8240 0x563c913f00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 -- 172.21.15.102:0/594841289 >> 172.21.15.102:0/594841289 conn(0x563c913e4560 msgr2=0x563c913ea260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f85d6ffd640 0 client.15840 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 -- 172.21.15.102:0/594841289 shutdown_connections 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 1 -- 172.21.15.102:0/594841289 wait complete. 2024-04-24T23:18:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.127+0000 7f8600f7d740 20 client.15840 trim_cache size 0 max 16384 2024-04-24T23:18:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.144+0000 7f8600f7d740 1 Processor -- start 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.144+0000 7f8600f7d740 1 -- start start 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.144+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 0x563c913f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 0x563c913feee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f8600f7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81b1da0 con 0x563c913f3230 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f8600f7d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81af670 con 0x563c913e8b60 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f8600f7d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81ad4e0 con 0x563c913e8240 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913ea0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56356/0 (socket says 172.21.15.102:56356) 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f7fff640 1 -- 172.21.15.102:0/2783400676 learned_addr learned my addr 172.21.15.102:0/2783400676 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f6ffd640 1 --2- 172.21.15.102:0/2783400676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 0x563c913feee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f7fff640 1 -- 172.21.15.102:0/2783400676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 msgr2=0x563c913f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f7fff640 1 --2- 172.21.15.102:0/2783400676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 0x563c913f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f7fff640 1 -- 172.21.15.102:0/2783400676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 msgr2=0x563c913feee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f7fff640 1 --2- 172.21.15.102:0/2783400676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 0x563c913feee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f7fff640 1 -- 172.21.15.102:0/2783400676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c913ee600 con 0x563c913f3230 2024-04-24T23:18:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85f7fff640 1 --2- 172.21.15.102:0/2783400676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913ea0a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f85f001ed40 tx=0x7f85f0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d17f491cb14deb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85dd7fa640 1 -- 172.21.15.102:0/2783400676 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0036020 con 0x563c913f3230 2024-04-24T23:18:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85dd7fa640 1 -- 172.21.15.102:0/2783400676 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f0025040 con 0x563c913f3230 2024-04-24T23:18:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.145+0000 7f85dd7fa640 1 -- 172.21.15.102:0/2783400676 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0032440 con 0x563c913f3230 2024-04-24T23:18:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 -- 172.21.15.102:0/2783400676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 msgr2=0x563c913ea0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 --2- 172.21.15.102:0/2783400676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913ea0a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f85f001ed40 tx=0x7f85f0005d30 comp rx=0 tx=0).stop 2024-04-24T23:18:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 -- 172.21.15.102:0/2783400676 shutdown_connections 2024-04-24T23:18:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 --2- 172.21.15.102:0/2783400676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913ea0a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 --2- 172.21.15.102:0/2783400676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 0x563c913feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 --2- 172.21.15.102:0/2783400676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 0x563c913f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 -- 172.21.15.102:0/2783400676 >> 172.21.15.102:0/2783400676 conn(0x563c913e4560 msgr2=0x563c913e9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 -- 172.21.15.102:0/2783400676 shutdown_connections 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 -- 172.21.15.102:0/2783400676 wait complete. 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.146+0000 7f8600f7d740 1 -- start start 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f8600f7d740 1 Processor -- start 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 0x563c913f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 0x563c913f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f8600f7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f8600f7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81adb30 con 0x563c913f3230 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f8600f7d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81aefa0 con 0x563c913e8b60 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f8600f7d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85f81ae200 con 0x563c913e8240 2024-04-24T23:18:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f8600f7d740 10 client.0 _ll_register_callbacks cb 0x7f85f0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56372/0 (socket says 172.21.15.102:56372) 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f7fff640 1 -- 172.21.15.102:0/687274929 learned_addr learned my addr 172.21.15.102:0/687274929 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f6ffd640 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 0x563c913f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f77fe640 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 0x563c913f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f7fff640 1 -- 172.21.15.102:0/687274929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 msgr2=0x563c913f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f7fff640 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 0x563c913f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f7fff640 1 -- 172.21.15.102:0/687274929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 msgr2=0x563c913f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f7fff640 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 0x563c913f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f7fff640 1 -- 172.21.15.102:0/687274929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c91468c70 con 0x563c913f3230 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f77fe640 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 0x563c913f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.147+0000 7f85f7fff640 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f7a50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f85f00094e0 tx=0x7f85f0010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45830b2226bdf38 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f85d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f85d6ffd640 1 -- 172.21.15.102:0/687274929 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f003f020 con 0x563c913f3230 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f85d6ffd640 1 -- 172.21.15.102:0/687274929 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f85f001e420 con 0x563c913f3230 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f8600f7d740 1 -- 172.21.15.102:0/687274929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f0025040 con 0x563c913f3230 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f85d6ffd640 1 -- 172.21.15.102:0/687274929 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85f0036030 con 0x563c913f3230 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f8600f7d740 10 client.15846 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f8600f7d740 1 -- 172.21.15.102:0/687274929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c91469f80 con 0x563c913f3230 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f85d6ffd640 1 -- 172.21.15.102:0/687274929 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f0025040 con 0x563c913f3230 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f85d6ffd640 1 -- 172.21.15.102:0/687274929 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f85f003bc20 con 0x563c913f3230 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f8600f7d740 10 client.15846 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f8600f7d740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f8600f7d740 20 client.15846 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f8600f7d740 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563c9146a530 0x563c9146c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f8600f7d740 4 client.15846 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.148+0000 7f8600f7d740 1 -- 172.21.15.102:0/687274929 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c9135a1f0 con 0x563c9146a530 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.149+0000 7f85f77fe640 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563c9146a530 0x563c9146c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.149+0000 7f85d6ffd640 10 client.15846 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.149+0000 7f85f77fe640 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563c9146a530 0x563c9146c950 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8558d2f2ffa26c9 server_cookie=6b43d9ae3a02ac91 in_seq=0 out_seq=0 2024-04-24T23:18:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f85d6ffd640 1 -- 172.21.15.102:0/687274929 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c9135a1f0 con 0x563c9146a530 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f85d6ffd640 10 client.15846 handle_command_reply: tid=0 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 client.15846 shutdown 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 -- 172.21.15.102:0/687274929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 msgr2=0x563c913f7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f7a50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f85f00094e0 tx=0x7f85f0010d90 comp rx=0 tx=0).stop 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 -- 172.21.15.102:0/687274929 shutdown_connections 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563c9146a530 0x563c9146c950 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c913f3230 0x563c913f7a50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f85d6ffd640 0 client.15846 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c913e8b60 0x563c913f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 --2- 172.21.15.102:0/687274929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c913e8240 0x563c913f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 -- 172.21.15.102:0/687274929 >> 172.21.15.102:0/687274929 conn(0x563c913e4560 msgr2=0x563c913ea220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 -- 172.21.15.102:0/687274929 shutdown_connections 2024-04-24T23:18:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 1 -- 172.21.15.102:0/687274929 wait complete. 2024-04-24T23:18:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.158+0000 7f8600f7d740 20 client.15846 trim_cache size 0 max 16384 2024-04-24T23:18:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:18:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:18:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:18:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 -- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85e00a2da0 msgr2=0x7f85e00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85e00a2da0 0x7f85e00a5260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85f0005c40 tx=0x7f85f0009a80 comp rx=0 tx=0).stop 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 -- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8100670 msgr2=0x7f85f80fd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8100670 0x7f85f80fd600 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f85e8004770 tx=0x7f85e80042f0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 -- 172.21.15.102:0/2145816972 shutdown_connections 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85f8106630 0x7f85f80fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85f8101030 0x7f85f80fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85e00a2da0 0x7f85e00a5260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 --2- 172.21.15.102:0/2145816972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85f8100670 0x7f85f80fd600 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 -- 172.21.15.102:0/2145816972 >> 172.21.15.102:0/2145816972 conn(0x7f85f80777c0 msgr2=0x7f85f8069130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 -- 172.21.15.102:0/2145816972 shutdown_connections 2024-04-24T23:18:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:33.159+0000 7f85fe190640 1 -- 172.21.15.102:0/2145816972 wait complete. 2024-04-24T23:18:33.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:32 smithi183 ceph-mon[33137]: pgmap v168: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:33.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:32 smithi102 ceph-mon[29216]: pgmap v168: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:33.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:32 smithi161 ceph-mon[32709]: pgmap v168: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:35.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:34 smithi183 ceph-mon[33137]: pgmap v169: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:35.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:34 smithi102 ceph-mon[29216]: pgmap v169: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:35.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:34 smithi161 ceph-mon[32709]: pgmap v169: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:37.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:36 smithi183 ceph-mon[33137]: pgmap v170: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:18:37.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:36 smithi102 ceph-mon[29216]: pgmap v170: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:18:37.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:36 smithi161 ceph-mon[32709]: pgmap v170: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:18:39.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:38 smithi183 ceph-mon[33137]: pgmap v171: 97 pgs: 97 active+clean; 7.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:38 smithi102 ceph-mon[29216]: pgmap v171: 97 pgs: 97 active+clean; 7.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:39.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:38 smithi161 ceph-mon[32709]: pgmap v171: 97 pgs: 97 active+clean; 7.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:18:41.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:40 smithi183 ceph-mon[33137]: pgmap v172: 97 pgs: 97 active+clean; 7.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:41.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:40 smithi102 ceph-mon[29216]: pgmap v172: 97 pgs: 97 active+clean; 7.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:41.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:40 smithi161 ceph-mon[32709]: pgmap v172: 97 pgs: 97 active+clean; 7.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:43.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:42 smithi183 ceph-mon[33137]: pgmap v173: 97 pgs: 97 active+clean; 7.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:43.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:42 smithi102 ceph-mon[29216]: pgmap v173: 97 pgs: 97 active+clean; 7.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:43.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:42 smithi161 ceph-mon[32709]: pgmap v173: 97 pgs: 97 active+clean; 7.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:45.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:44 smithi183 ceph-mon[33137]: pgmap v174: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:45.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:44 smithi102 ceph-mon[29216]: pgmap v174: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:45.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:44 smithi161 ceph-mon[32709]: pgmap v174: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.248+0000 7f3c271c1640 1 -- 172.21.15.102:0/2494880215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20101790 msgr2=0x7f3c2010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.248+0000 7f3c271c1640 1 --2- 172.21.15.102:0/2494880215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20101790 0x7f3c2010d4a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009ca0 tx=0x7f3c1802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.248+0000 7f3c271c1640 1 -- 172.21.15.102:0/2494880215 shutdown_connections 2024-04-24T23:18:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.248+0000 7f3c271c1640 1 --2- 172.21.15.102:0/2494880215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20101790 0x7f3c2010d4a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.248+0000 7f3c271c1640 1 --2- 172.21.15.102:0/2494880215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c20100dd0 0x7f3c20101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.248+0000 7f3c271c1640 1 --2- 172.21.15.102:0/2494880215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c20105610 0x7f3c201059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.248+0000 7f3c271c1640 1 -- 172.21.15.102:0/2494880215 >> 172.21.15.102:0/2494880215 conn(0x7f3c20076b60 msgr2=0x7f3c200fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.248+0000 7f3c271c1640 1 -- 172.21.15.102:0/2494880215 shutdown_connections 2024-04-24T23:18:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.248+0000 7f3c271c1640 1 -- 172.21.15.102:0/2494880215 wait complete. 2024-04-24T23:18:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c271c1640 1 Processor -- start 2024-04-24T23:18:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c271c1640 1 -- start start 2024-04-24T23:18:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c271c1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20100dd0 0x7f3c20197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c271c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c20101790 0x7f3c20198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c271c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c20105610 0x7f3c2019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c271c1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20110900 con 0x7f3c20100dd0 2024-04-24T23:18:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c271c1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20110780 con 0x7f3c20105610 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c271c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20110a80 con 0x7f3c20101790 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c24f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20100dd0 0x7f3c20197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c24f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20100dd0 0x7f3c20197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44856/0 (socket says 172.21.15.102:44856) 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c24f5d640 1 -- 172.21.15.102:0/1295799397 learned_addr learned my addr 172.21.15.102:0/1295799397 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.249+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c20101790 0x7f3c20198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c2575e640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c20105610 0x7f3c2019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c24f5d640 1 -- 172.21.15.102:0/1295799397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c20101790 msgr2=0x7f3c20198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c20101790 0x7f3c20198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c24f5d640 1 -- 172.21.15.102:0/1295799397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c20105610 msgr2=0x7f3c2019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c20105610 0x7f3c2019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c24f5d640 1 -- 172.21.15.102:0/1295799397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2019ce30 con 0x7f3c20100dd0 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20100dd0 0x7f3c20197e40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400b4a0 tx=0x7f3c1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c1dffb640 1 -- 172.21.15.102:0/1295799397 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14014070 con 0x7f3c20100dd0 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c1dffb640 1 -- 172.21.15.102:0/1295799397 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c14010ac0 con 0x7f3c20100dd0 2024-04-24T23:18:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c271c1640 1 -- 172.21.15.102:0/1295799397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2019d120 con 0x7f3c20100dd0 2024-04-24T23:18:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c1dffb640 1 -- 172.21.15.102:0/1295799397 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14017c30 con 0x7f3c20100dd0 2024-04-24T23:18:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.250+0000 7f3c271c1640 1 -- 172.21.15.102:0/1295799397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c201a49b0 con 0x7f3c20100dd0 2024-04-24T23:18:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.252+0000 7f3c1dffb640 1 -- 172.21.15.102:0/1295799397 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3c14017070 con 0x7f3c20100dd0 2024-04-24T23:18:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.253+0000 7f3c1dffb640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3c100a2e20 0x7f3c100a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.253+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3c100a2e20 0x7f3c100a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.253+0000 7f3c1dffb640 1 -- 172.21.15.102:0/1295799397 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f3c140ce670 con 0x7f3c20100dd0 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.253+0000 7f3c29fab740 1 Processor -- start 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.253+0000 7f3c29fab740 1 -- start start 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.253+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a45620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a4fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a52620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c29fab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5610d79a9cb0 con 0x5610d7a50230 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c29fab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5610d79525b0 con 0x5610d7a45b60 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c29fab740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5610d79a5c70 con 0x5610d7a45240 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3c100a2e20 0x7f3c100a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c20104ee0 tx=0x7f3c0c00b410 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c2575e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a52620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c2575e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a52620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44872/0 (socket says 172.21.15.102:44872) 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c2575e640 1 -- 172.21.15.102:0/1278618323 learned_addr learned my addr 172.21.15.102:0/1278618323 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a4fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a45620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c24f5d640 1 -- 172.21.15.102:0/1278618323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 msgr2=0x5610d7a4fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a4fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c24f5d640 1 -- 172.21.15.102:0/1278618323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 msgr2=0x5610d7a52620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a52620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c24f5d640 1 -- 172.21.15.102:0/1278618323 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d76be210 con 0x5610d7a45240 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c2575e640 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a52620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a45620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c140cbec0 tx=0x7f3c1409a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183d8a867fadbc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.254+0000 7f3c0affd640 1 -- 172.21.15.102:0/1278618323 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1401e070 con 0x5610d7a45240 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c0affd640 1 -- 172.21.15.102:0/1278618323 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c1409f040 con 0x5610d7a45240 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c0affd640 1 -- 172.21.15.102:0/1278618323 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14010c70 con 0x5610d7a45240 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c29fab740 1 -- 172.21.15.102:0/1278618323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 msgr2=0x5610d7a45620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a45620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c140cbec0 tx=0x7f3c1409a8d0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c29fab740 1 -- 172.21.15.102:0/1278618323 shutdown_connections 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a52620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a4fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1278618323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a45620 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c29fab740 1 -- 172.21.15.102:0/1278618323 >> 172.21.15.102:0/1278618323 conn(0x5610d7a41560 msgr2=0x5610d7a41940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c29fab740 1 -- 172.21.15.102:0/1278618323 shutdown_connections 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.255+0000 7f3c29fab740 1 -- 172.21.15.102:0/1278618323 wait complete. 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c29fab740 1 -- start start 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c29fab740 1 Processor -- start 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a4d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45b60 0x5610d7a4d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a50230 0x5610d7a4dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c29fab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5610d7a53020 con 0x5610d7a45b60 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c29fab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5610d7965b50 con 0x5610d7a50230 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c29fab740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5610d7a531a0 con 0x5610d7a45240 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c29fab740 10 client.0 _ll_register_callbacks cb 0x5610d79a0ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c1ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45b60 0x5610d7a4d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c1ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45b60 0x5610d7a4d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44878/0 (socket says 172.21.15.102:44878) 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c1ffff640 1 -- 172.21.15.102:0/1391724234 learned_addr learned my addr 172.21.15.102:0/1391724234 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a4d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.256+0000 7f3c2575e640 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a50230 0x5610d7a4dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3c1ffff640 1 -- 172.21.15.102:0/1391724234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 msgr2=0x5610d7a4d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a4d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3c1ffff640 1 -- 172.21.15.102:0/1391724234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a50230 msgr2=0x5610d7a4dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a50230 0x5610d7a4dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3c1ffff640 1 -- 172.21.15.102:0/1391724234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d778a620 con 0x5610d7a45b60 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45b60 0x5610d7a4d710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c011860 tx=0x7f3c0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e5d0c8584870ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3bfffff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3bfffff640 1 -- 172.21.15.102:0/1391724234 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c01e8c0 con 0x5610d7a45b60 2024-04-24T23:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3bfffff640 1 -- 172.21.15.102:0/1391724234 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f3c0c01ea60 con 0x5610d7a45b60 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3c29fab740 1 -- 172.21.15.102:0/1391724234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c1409f040 con 0x5610d7a45b60 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3bfffff640 1 -- 172.21.15.102:0/1391724234 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c0c01ec60 con 0x5610d7a45b60 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3c29fab740 10 client.15858 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.257+0000 7f3c29fab740 1 -- 172.21.15.102:0/1391724234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610d7a5f280 con 0x5610d7a45b60 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3bfffff640 1 -- 172.21.15.102:0/1391724234 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c1409f040 con 0x5610d7a45b60 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3bfffff640 1 -- 172.21.15.102:0/1391724234 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3c0c01f430 con 0x5610d7a45b60 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3c29fab740 10 client.15858 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3c29fab740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3c29fab740 20 client.15858 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5610d7a607e0 0x5610d7a62bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3c29fab740 4 client.15858 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3c29fab740 1 -- 172.21.15.102:0/1391724234 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610d79b71f0 con 0x5610d7a607e0 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5610d7a607e0 0x5610d7a62bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3bfffff640 10 client.15858 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.258+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5610d7a607e0 0x5610d7a62bd0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e30af3d83c07714 server_cookie=e0fe6bb4baac3899 in_seq=0 out_seq=0 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3bfffff640 1 -- 172.21.15.102:0/1391724234 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610d79b71f0 con 0x5610d7a607e0 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3bfffff640 10 client.15858 handle_command_reply: tid=0 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3c29fab740 1 client.15858 shutdown 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3c29fab740 1 -- 172.21.15.102:0/1391724234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45b60 msgr2=0x5610d7a4d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45b60 0x5610d7a4d710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c011860 tx=0x7f3c0c004730 comp rx=0 tx=0).stop 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3c29fab740 1 -- 172.21.15.102:0/1391724234 shutdown_connections 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a50230 0x5610d7a4dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5610d7a607e0 0x5610d7a62bd0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45b60 0x5610d7a4d710 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3c29fab740 1 --2- 172.21.15.102:0/1391724234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a4d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3bfffff640 0 client.15858 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.259+0000 7f3c29fab740 1 -- 172.21.15.102:0/1391724234 >> 172.21.15.102:0/1391724234 conn(0x5610d7a41560 msgr2=0x5610d7a47360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.260+0000 7f3c29fab740 1 -- 172.21.15.102:0/1391724234 shutdown_connections 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.260+0000 7f3c29fab740 1 -- 172.21.15.102:0/1391724234 wait complete. 2024-04-24T23:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.260+0000 7f3c29fab740 20 client.15858 trim_cache size 0 max 16384 2024-04-24T23:18:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c29fab740 1 Processor -- start 2024-04-24T23:18:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c29fab740 1 -- start start 2024-04-24T23:18:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45240 0x5610d7a5bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a479a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a50230 0x5610d7a684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c29fab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201afcb0 con 0x5610d7a45240 2024-04-24T23:18:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c29fab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201ad580 con 0x5610d7a45b60 2024-04-24T23:18:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c29fab740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201ab3f0 con 0x5610d7a50230 2024-04-24T23:18:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c24f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45240 0x5610d7a5bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c24f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45240 0x5610d7a5bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44890/0 (socket says 172.21.15.102:44890) 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c24f5d640 1 -- 172.21.15.102:0/3842805261 learned_addr learned my addr 172.21.15.102:0/3842805261 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/3842805261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c24f5d640 1 -- 172.21.15.102:0/3842805261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a50230 msgr2=0x5610d7a684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/3842805261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a50230 0x5610d7a684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c24f5d640 1 -- 172.21.15.102:0/3842805261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 msgr2=0x5610d7a479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/3842805261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c24f5d640 1 -- 172.21.15.102:0/3842805261 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d7a45990 con 0x5610d7a45240 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/3842805261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.279+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/3842805261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45240 0x5610d7a5bee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3c140cc3b0 tx=0x7f3c1409ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e0d2dc26e53c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c0a7fc640 1 -- 172.21.15.102:0/3842805261 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c140b7070 con 0x5610d7a45240 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c0a7fc640 1 -- 172.21.15.102:0/3842805261 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c140cc030 con 0x5610d7a45240 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c0a7fc640 1 -- 172.21.15.102:0/3842805261 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1401e040 con 0x5610d7a45240 2024-04-24T23:18:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c29fab740 1 -- 172.21.15.102:0/3842805261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45240 msgr2=0x5610d7a5bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c29fab740 1 --2- 172.21.15.102:0/3842805261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45240 0x5610d7a5bee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3c140cc3b0 tx=0x7f3c1409ae10 comp rx=0 tx=0).stop 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c29fab740 1 -- 172.21.15.102:0/3842805261 shutdown_connections 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c29fab740 1 --2- 172.21.15.102:0/3842805261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a50230 0x5610d7a684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c29fab740 1 --2- 172.21.15.102:0/3842805261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c29fab740 1 --2- 172.21.15.102:0/3842805261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a45240 0x5610d7a5bee0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c29fab740 1 -- 172.21.15.102:0/3842805261 >> 172.21.15.102:0/3842805261 conn(0x5610d7a41560 msgr2=0x5610d7a405c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c29fab740 1 -- 172.21.15.102:0/3842805261 shutdown_connections 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.280+0000 7f3c29fab740 1 -- 172.21.15.102:0/3842805261 wait complete. 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c29fab740 1 -- start start 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c29fab740 1 Processor -- start 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c29fab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a4abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c29fab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201aba40 con 0x5610d7a50230 2024-04-24T23:18:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c29fab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201aceb0 con 0x5610d7a45b60 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c29fab740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201ac110 con 0x5610d7a45240 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c29fab740 10 client.0 _ll_register_callbacks cb 0x7f3c0c00bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c2575e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a4abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c2575e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a4abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44894/0 (socket says 172.21.15.102:44894) 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c2575e640 1 -- 172.21.15.102:0/78584855 learned_addr learned my addr 172.21.15.102:0/78584855 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.281+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c1ffff640 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c2575e640 1 -- 172.21.15.102:0/78584855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 msgr2=0x5610d7a4a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c2575e640 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a4a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c2575e640 1 -- 172.21.15.102:0/78584855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 msgr2=0x5610d7a4a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c2575e640 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a4a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c2575e640 1 -- 172.21.15.102:0/78584855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d7ac5fa0 con 0x5610d7a50230 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c2575e640 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a4abb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005cc0 tx=0x7f3c18031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e80bda7f2448df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3bfffff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3bfffff640 1 -- 172.21.15.102:0/78584855 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18003b70 con 0x5610d7a50230 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3bfffff640 1 -- 172.21.15.102:0/78584855 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f3c18003d10 con 0x5610d7a50230 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c29fab740 1 -- 172.21.15.102:0/78584855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c140cc030 con 0x5610d7a50230 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3bfffff640 1 -- 172.21.15.102:0/78584855 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c18053050 con 0x5610d7a50230 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c29fab740 10 client.15864 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3c29fab740 1 -- 172.21.15.102:0/78584855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610d7ac71e0 con 0x5610d7a50230 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.282+0000 7f3bfffff640 1 -- 172.21.15.102:0/78584855 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c140cc030 con 0x5610d7a50230 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3bfffff640 1 -- 172.21.15.102:0/78584855 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3c1804e030 con 0x5610d7a50230 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3c29fab740 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3c29fab740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3c29fab740 20 client.15864 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3c29fab740 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5610d7ac79e0 0x5610d7ac9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3c29fab740 4 client.15864 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:18:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3c29fab740 1 -- 172.21.15.102:0/78584855 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610d79b1540 con 0x5610d7ac79e0 2024-04-24T23:18:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5610d7ac79e0 0x5610d7ac9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3bfffff640 10 client.15864 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.283+0000 7f3c24f5d640 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5610d7ac79e0 0x5610d7ac9e00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99c1070a389b14f server_cookie=1ead37a4818ba42b in_seq=0 out_seq=0 2024-04-24T23:18:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3bfffff640 1 -- 172.21.15.102:0/78584855 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610d79b1540 con 0x5610d7ac79e0 2024-04-24T23:18:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3bfffff640 10 client.15864 handle_command_reply: tid=0 2024-04-24T23:18:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3c29fab740 1 client.15864 shutdown 2024-04-24T23:18:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3c29fab740 1 -- 172.21.15.102:0/78584855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 msgr2=0x5610d7a4abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3c29fab740 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a4abb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005cc0 tx=0x7f3c18031c60 comp rx=0 tx=0).stop 2024-04-24T23:18:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3c29fab740 1 -- 172.21.15.102:0/78584855 shutdown_connections 2024-04-24T23:18:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3c29fab740 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5610d7ac79e0 0x5610d7ac9e00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3c29fab740 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610d7a50230 0x5610d7a4abb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3c29fab740 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610d7a45b60 0x5610d7a4a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3bfffff640 0 client.15864 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3c29fab740 1 --2- 172.21.15.102:0/78584855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610d7a45240 0x5610d7a4a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.290+0000 7f3c29fab740 1 -- 172.21.15.102:0/78584855 >> 172.21.15.102:0/78584855 conn(0x5610d7a41560 msgr2=0x5610d7a4d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c29fab740 1 -- 172.21.15.102:0/78584855 shutdown_connections 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c29fab740 1 -- 172.21.15.102:0/78584855 wait complete. 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c29fab740 20 client.15864 trim_cache size 0 max 16384 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:18:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 -- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3c100a2e20 msgr2=0x7f3c100a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3c100a2e20 0x7f3c100a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c20104ee0 tx=0x7f3c0c00b410 comp rx=0 tx=0).stop 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 -- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20100dd0 msgr2=0x7f3c20197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20100dd0 0x7f3c20197e40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400b4a0 tx=0x7f3c1400b960 comp rx=0 tx=0).stop 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 -- 172.21.15.102:0/1295799397 shutdown_connections 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c20105610 0x7f3c2019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c20101790 0x7f3c20198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3c100a2e20 0x7f3c100a52e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 --2- 172.21.15.102:0/1295799397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c20100dd0 0x7f3c20197e40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.291+0000 7f3c271c1640 1 -- 172.21.15.102:0/1295799397 >> 172.21.15.102:0/1295799397 conn(0x7f3c20076b60 msgr2=0x7f3c200ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.292+0000 7f3c271c1640 1 -- 172.21.15.102:0/1295799397 shutdown_connections 2024-04-24T23:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:46.292+0000 7f3c271c1640 1 -- 172.21.15.102:0/1295799397 wait complete. 2024-04-24T23:18:47.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:46 smithi183 ceph-mon[33137]: pgmap v175: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:47.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:46 smithi102 ceph-mon[29216]: pgmap v175: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:47.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:46 smithi161 ceph-mon[32709]: pgmap v175: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:49.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:48 smithi183 ceph-mon[33137]: pgmap v176: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:18:49.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:48 smithi102 ceph-mon[29216]: pgmap v176: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:18:49.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:48 smithi161 ceph-mon[32709]: pgmap v176: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:18:51.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:50 smithi183 ceph-mon[33137]: pgmap v177: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:51.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:50 smithi102 ceph-mon[29216]: pgmap v177: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:51.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:50 smithi161 ceph-mon[32709]: pgmap v177: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:53.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:52 smithi183 ceph-mon[33137]: pgmap v178: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:53.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:52 smithi102 ceph-mon[29216]: pgmap v178: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:53.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:52 smithi161 ceph-mon[32709]: pgmap v178: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:55.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:54 smithi183 ceph-mon[33137]: pgmap v179: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:55.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:54 smithi102 ceph-mon[29216]: pgmap v179: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:55.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:54 smithi161 ceph-mon[32709]: pgmap v179: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:57.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:56 smithi183 ceph-mon[33137]: pgmap v180: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:56 smithi102 ceph-mon[29216]: pgmap v180: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:56 smithi161 ceph-mon[32709]: pgmap v180: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:18:59.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:18:58 smithi183 ceph-mon[33137]: pgmap v181: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:59.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:18:58 smithi102 ceph-mon[29216]: pgmap v181: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:59.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:18:58 smithi161 ceph-mon[32709]: pgmap v181: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:18:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.384+0000 7f5bbe512640 1 -- 172.21.15.102:0/2537935330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8106630 msgr2=0x7f5bb8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.384+0000 7f5bbe512640 1 --2- 172.21.15.102:0/2537935330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8106630 0x7f5bb8106a10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009930 tx=0x7f5ba802a420 comp rx=0 tx=0).stop 2024-04-24T23:18:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 -- 172.21.15.102:0/2537935330 shutdown_connections 2024-04-24T23:18:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 --2- 172.21.15.102:0/2537935330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb8101030 0x7f5bb810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 --2- 172.21.15.102:0/2537935330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb8100670 0x7f5bb8100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 --2- 172.21.15.102:0/2537935330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8106630 0x7f5bb8106a10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 -- 172.21.15.102:0/2537935330 >> 172.21.15.102:0/2537935330 conn(0x7f5bb80777c0 msgr2=0x7f5bb8077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 -- 172.21.15.102:0/2537935330 shutdown_connections 2024-04-24T23:18:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 -- 172.21.15.102:0/2537935330 wait complete. 2024-04-24T23:18:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 Processor -- start 2024-04-24T23:18:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 -- start start 2024-04-24T23:18:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.385+0000 7f5bbe512640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8100670 0x7f5bb80fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bbe512640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb8101030 0x7f5bb80fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bbe512640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb8106630 0x7f5bb80fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bbe512640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81109c0 con 0x7f5bb8100670 2024-04-24T23:18:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bbe512640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8110840 con 0x7f5bb8106630 2024-04-24T23:18:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bbe512640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8110b40 con 0x7f5bb8101030 2024-04-24T23:18:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8100670 0x7f5bb80fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8100670 0x7f5bb80fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38502/0 (socket says 172.21.15.102:38502) 2024-04-24T23:18:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb7fff640 1 -- 172.21.15.102:0/432022277 learned_addr learned my addr 172.21.15.102:0/432022277 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb8101030 0x7f5bb80fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bbcaaf640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb8106630 0x7f5bb80fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb7fff640 1 -- 172.21.15.102:0/432022277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb8101030 msgr2=0x7f5bb80fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb8101030 0x7f5bb80fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb7fff640 1 -- 172.21.15.102:0/432022277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb8106630 msgr2=0x7f5bb80fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb8106630 0x7f5bb80fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb7fff640 1 -- 172.21.15.102:0/432022277 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb81a65b0 con 0x7f5bb8100670 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8100670 0x7f5bb80fd600 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8031440 tx=0x7f5ba8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb57fa640 1 -- 172.21.15.102:0/432022277 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba803f070 con 0x7f5bb8100670 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bbe512640 1 -- 172.21.15.102:0/432022277 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb81a6840 con 0x7f5bb8100670 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb57fa640 1 -- 172.21.15.102:0/432022277 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba8031d60 con 0x7f5bb8100670 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.386+0000 7f5bb57fa640 1 -- 172.21.15.102:0/432022277 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8035030 con 0x7f5bb8100670 2024-04-24T23:18:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.387+0000 7f5bbe512640 1 -- 172.21.15.102:0/432022277 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb81a6d30 con 0x7f5bb8100670 2024-04-24T23:18:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.389+0000 7f5bc1216740 1 Processor -- start 2024-04-24T23:18:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.389+0000 7f5bb57fa640 1 -- 172.21.15.102:0/432022277 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f5ba8042070 con 0x7f5bb8100670 2024-04-24T23:18:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.389+0000 7f5bc1216740 1 -- start start 2024-04-24T23:18:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 0x555dea7b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bc1216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555dea70bcb0 con 0x555dea7a7b60 2024-04-24T23:18:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bc1216740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555dea6b45b0 con 0x555dea7a7240 2024-04-24T23:18:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bc1216740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555dea707c70 con 0x555dea7b2230 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bb7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bbcaaf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bbcaaf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7a7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:50734/0 (socket says 172.21.15.102:50734) 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bbcaaf640 1 -- 172.21.15.102:0/2679000907 learned_addr learned my addr 172.21.15.102:0/2679000907 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/2679000907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 0x555dea7b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bbcaaf640 1 -- 172.21.15.102:0/2679000907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 msgr2=0x555dea7b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bbcaaf640 1 --2- 172.21.15.102:0/2679000907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 0x555dea7b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bbcaaf640 1 -- 172.21.15.102:0/2679000907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 msgr2=0x555dea7b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bbcaaf640 1 --2- 172.21.15.102:0/2679000907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bbcaaf640 1 -- 172.21.15.102:0/2679000907 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dea420210 con 0x555dea7a7240 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/2679000907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bb57fa640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5ba00a2da0 0x7f5ba00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5ba00a2da0 0x7f5ba00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bb57fa640 1 -- 172.21.15.102:0/432022277 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5ba80ef960 con 0x7f5bb8100670 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.390+0000 7f5bbcaaf640 1 --2- 172.21.15.102:0/2679000907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7a7620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000a430 tx=0x7f5bb0009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79298990c768bb71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5b9e7fc640 1 -- 172.21.15.102:0/2679000907 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0013790 con 0x555dea7a7240 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5b9e7fc640 1 -- 172.21.15.102:0/2679000907 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bb0019070 con 0x555dea7a7240 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5b9e7fc640 1 -- 172.21.15.102:0/2679000907 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0014ca0 con 0x555dea7a7240 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bc1216740 1 -- 172.21.15.102:0/2679000907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 msgr2=0x555dea7a7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5ba00a2da0 0x7f5ba00a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x555dea7a46f0 tx=0x7f5bac008040 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bc1216740 1 --2- 172.21.15.102:0/2679000907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7a7620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000a430 tx=0x7f5bb0009650 comp rx=0 tx=0).stop 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bc1216740 1 -- 172.21.15.102:0/2679000907 shutdown_connections 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bc1216740 1 --2- 172.21.15.102:0/2679000907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 0x555dea7b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bc1216740 1 --2- 172.21.15.102:0/2679000907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bc1216740 1 --2- 172.21.15.102:0/2679000907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7a7620 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bc1216740 1 -- 172.21.15.102:0/2679000907 >> 172.21.15.102:0/2679000907 conn(0x555dea7a3560 msgr2=0x555dea7a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bc1216740 1 -- 172.21.15.102:0/2679000907 shutdown_connections 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.391+0000 7f5bc1216740 1 -- 172.21.15.102:0/2679000907 wait complete. 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bc1216740 1 -- start start 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bc1216740 1 Processor -- start 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 0x555dea7af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 0x555dea7afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bc1216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555dea7b5020 con 0x555dea7a7b60 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bc1216740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555dea6c7b50 con 0x555dea7b2230 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bc1216740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555dea7b51a0 con 0x555dea7a7240 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bc1216740 10 client.0 _ll_register_callbacks cb 0x555dea702ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bb77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bb77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38524/0 (socket says 172.21.15.102:38524) 2024-04-24T23:18:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bb77fe640 1 -- 172.21.15.102:0/4210977742 learned_addr learned my addr 172.21.15.102:0/4210977742 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bbcaaf640 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 0x555dea7afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.392+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 0x555dea7af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bb77fe640 1 -- 172.21.15.102:0/4210977742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 msgr2=0x555dea7af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 0x555dea7af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bb77fe640 1 -- 172.21.15.102:0/4210977742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 msgr2=0x555dea7afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 0x555dea7afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bb77fe640 1 -- 172.21.15.102:0/4210977742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dea4ec620 con 0x555dea7a7b60 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 0x555dea7af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7af6d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0156c0 tx=0x7f5bac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a449593605dd2151 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5b977fe640 1 -- 172.21.15.102:0/4210977742 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac025020 con 0x555dea7a7b60 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5b977fe640 1 -- 172.21.15.102:0/4210977742 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5bac021440 con 0x555dea7a7b60 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bc1216740 1 -- 172.21.15.102:0/4210977742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb0019070 con 0x555dea7a7b60 2024-04-24T23:18:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5b977fe640 1 -- 172.21.15.102:0/4210977742 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5bac030050 con 0x555dea7a7b60 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bc1216740 10 client.15876 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5bc1216740 1 -- 172.21.15.102:0/4210977742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555dea7c1240 con 0x555dea7a7b60 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.393+0000 7f5b977fe640 1 -- 172.21.15.102:0/4210977742 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bb0019070 con 0x555dea7a7b60 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5b977fe640 1 -- 172.21.15.102:0/4210977742 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5bac0146f0 con 0x555dea7a7b60 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5bc1216740 10 client.15876 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5bc1216740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5bc1216740 20 client.15876 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555dea7c2700 0x555dea7c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5bc1216740 4 client.15876 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5bc1216740 1 -- 172.21.15.102:0/4210977742 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dea7191f0 con 0x555dea7c2700 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555dea7c2700 0x555dea7c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5b977fe640 10 client.15876 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.394+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555dea7c2700 0x555dea7c4b50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba3d6e3f45379f8 server_cookie=f1aa249c6bee74c3 in_seq=0 out_seq=0 2024-04-24T23:18:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.395+0000 7f5b977fe640 1 -- 172.21.15.102:0/4210977742 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555dea7191f0 con 0x555dea7c2700 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.395+0000 7f5b977fe640 10 client.15876 handle_command_reply: tid=0 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.395+0000 7f5bc1216740 1 client.15876 shutdown 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.395+0000 7f5bc1216740 1 -- 172.21.15.102:0/4210977742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 msgr2=0x555dea7af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.395+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7af6d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0156c0 tx=0x7f5bac0046c0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5bc1216740 1 -- 172.21.15.102:0/4210977742 shutdown_connections 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 0x555dea7afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555dea7c2700 0x555dea7c4b50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7af6d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4210977742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 0x555dea7af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5b977fe640 0 client.15876 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5bc1216740 1 -- 172.21.15.102:0/4210977742 >> 172.21.15.102:0/4210977742 conn(0x555dea7a3560 msgr2=0x555dea7a9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5bc1216740 1 -- 172.21.15.102:0/4210977742 shutdown_connections 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5bc1216740 1 -- 172.21.15.102:0/4210977742 wait complete. 2024-04-24T23:18:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.396+0000 7f5bc1216740 20 client.15876 trim_cache size 0 max 16384 2024-04-24T23:18:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bc1216740 1 Processor -- start 2024-04-24T23:18:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bc1216740 1 -- start start 2024-04-24T23:18:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 0x555dea7a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bc1216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81b1ea0 con 0x555dea7a7b60 2024-04-24T23:18:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bc1216740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81af770 con 0x555dea7a7240 2024-04-24T23:18:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bc1216740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81ad5e0 con 0x555dea7b2230 2024-04-24T23:18:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7bdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38536/0 (socket says 172.21.15.102:38536) 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb77fe640 1 -- 172.21.15.102:0/2971637294 learned_addr learned my addr 172.21.15.102:0/2971637294 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bbcaaf640 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 0x555dea7a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb77fe640 1 -- 172.21.15.102:0/2971637294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 msgr2=0x555dea7a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 0x555dea7a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb77fe640 1 -- 172.21.15.102:0/2971637294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 msgr2=0x555dea7b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb77fe640 1 -- 172.21.15.102:0/2971637294 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dea7ae160 con 0x555dea7a7b60 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bbcaaf640 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 0x555dea7a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.411+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7bdee0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5bac008010 tx=0x7f5bac009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29df10bd4044a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5b9dffb640 1 -- 172.21.15.102:0/2971637294 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac025040 con 0x555dea7a7b60 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5b9dffb640 1 -- 172.21.15.102:0/2971637294 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bac039070 con 0x555dea7a7b60 2024-04-24T23:18:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5b9dffb640 1 -- 172.21.15.102:0/2971637294 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac020a30 con 0x555dea7a7b60 2024-04-24T23:18:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5bc1216740 1 -- 172.21.15.102:0/2971637294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 msgr2=0x555dea7bdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5bc1216740 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7bdee0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5bac008010 tx=0x7f5bac009790 comp rx=0 tx=0).stop 2024-04-24T23:18:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5bc1216740 1 -- 172.21.15.102:0/2971637294 shutdown_connections 2024-04-24T23:18:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5bc1216740 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7b2230 0x555dea7a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5bc1216740 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7bdee0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5bc1216740 1 --2- 172.21.15.102:0/2971637294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7a7240 0x555dea7b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5bc1216740 1 -- 172.21.15.102:0/2971637294 >> 172.21.15.102:0/2971637294 conn(0x555dea7a3560 msgr2=0x555dea7a25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.412+0000 7f5bc1216740 1 -- 172.21.15.102:0/2971637294 shutdown_connections 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 1 -- 172.21.15.102:0/2971637294 wait complete. 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 1 -- start start 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 1 Processor -- start 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 0x555dea7ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 0x555dea7ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81adc30 con 0x555dea7a7b60 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81af0a0 con 0x555dea7b2230 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81ae300 con 0x555dea7a7240 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bc1216740 10 client.0 _ll_register_callbacks cb 0x7f5bac0096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bb77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bb77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38542/0 (socket says 172.21.15.102:38542) 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bb77fe640 1 -- 172.21.15.102:0/4154960184 learned_addr learned my addr 172.21.15.102:0/4154960184 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:18:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bbcaaf640 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 0x555dea7ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.413+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 0x555dea7ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5bb77fe640 1 -- 172.21.15.102:0/4154960184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 msgr2=0x555dea7ba940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 0x555dea7ba940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5bb77fe640 1 -- 172.21.15.102:0/4154960184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 msgr2=0x555dea7ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 0x555dea7ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5bb77fe640 1 -- 172.21.15.102:0/4154960184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dea827cd0 con 0x555dea7a7b60 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5bb77fe640 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7abca0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5bac014c30 tx=0x7f5bac009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb17ebcf32c2a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5b977fe640 1 -- 172.21.15.102:0/4154960184 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac025070 con 0x555dea7a7b60 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5b977fe640 1 -- 172.21.15.102:0/4154960184 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f5bac014730 con 0x555dea7a7b60 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5bc1216740 1 -- 172.21.15.102:0/4154960184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bac039070 con 0x555dea7a7b60 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5b977fe640 1 -- 172.21.15.102:0/4154960184 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5bac0207b0 con 0x555dea7a7b60 2024-04-24T23:18:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5bc1216740 10 client.15882 fetch_fsmap learned FSMap version 26 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5bc1216740 1 -- 172.21.15.102:0/4154960184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555dea828d70 con 0x555dea7a7b60 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.414+0000 7f5b977fe640 1 -- 172.21.15.102:0/4154960184 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bac039070 con 0x555dea7a7b60 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5b977fe640 1 -- 172.21.15.102:0/4154960184 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5bac045030 con 0x555dea7a7b60 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5bc1216740 10 client.15882 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5bc1216740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5bc1216740 20 client.15882 populate_metadata read hostname 'smithi102' 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555dea829430 0x555dea82b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5bc1216740 4 client.15882 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5bc1216740 1 -- 172.21.15.102:0/4154960184 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555dea713540 con 0x555dea829430 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555dea829430 0x555dea82b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5b977fe640 10 client.15882 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.415+0000 7f5bb7fff640 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555dea829430 0x555dea82b850 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7beb122b9c418c13 server_cookie=d03e485d110be7fa in_seq=0 out_seq=0 2024-04-24T23:18:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5b977fe640 1 -- 172.21.15.102:0/4154960184 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555dea713540 con 0x555dea829430 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5b977fe640 10 client.15882 handle_command_reply: tid=0 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 client.15882 shutdown 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 -- 172.21.15.102:0/4154960184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 msgr2=0x555dea7abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7abca0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5bac014c30 tx=0x7f5bac009180 comp rx=0 tx=0).stop 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 -- 172.21.15.102:0/4154960184 shutdown_connections 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555dea7b2230 0x555dea7ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555dea829430 0x555dea82b850 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555dea7a7b60 0x555dea7abca0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5b977fe640 0 client.15882 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 --2- 172.21.15.102:0/4154960184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555dea7a7240 0x555dea7ba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 -- 172.21.15.102:0/4154960184 >> 172.21.15.102:0/4154960184 conn(0x555dea7a3560 msgr2=0x555dea7a25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 -- 172.21.15.102:0/4154960184 shutdown_connections 2024-04-24T23:18:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 1 -- 172.21.15.102:0/4154960184 wait complete. 2024-04-24T23:18:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.429+0000 7f5bc1216740 20 client.15882 trim_cache size 0 max 16384 2024-04-24T23:18:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:18:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:18:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:18:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:18:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 -- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5ba00a2da0 msgr2=0x7f5ba00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5ba00a2da0 0x7f5ba00a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x555dea7a46f0 tx=0x7f5bac008040 comp rx=0 tx=0).stop 2024-04-24T23:18:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 -- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8100670 msgr2=0x7f5bb80fd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8100670 0x7f5bb80fd600 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8031440 tx=0x7f5ba8003d50 comp rx=0 tx=0).stop 2024-04-24T23:18:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 -- 172.21.15.102:0/432022277 shutdown_connections 2024-04-24T23:18:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb8106630 0x7f5bb80fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb8101030 0x7f5bb80fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5ba00a2da0 0x7f5ba00a5260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 --2- 172.21.15.102:0/432022277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb8100670 0x7f5bb80fd600 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 -- 172.21.15.102:0/432022277 >> 172.21.15.102:0/432022277 conn(0x7f5bb80777c0 msgr2=0x7f5bb80691f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 -- 172.21.15.102:0/432022277 shutdown_connections 2024-04-24T23:18:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:18:59.430+0000 7f5bbe512640 1 -- 172.21.15.102:0/432022277 wait complete. 2024-04-24T23:19:01.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:00 smithi183 ceph-mon[33137]: pgmap v182: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:19:01.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:00 smithi102 ceph-mon[29216]: pgmap v182: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:19:01.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:00 smithi161 ceph-mon[32709]: pgmap v182: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:19:03.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:02 smithi183 ceph-mon[33137]: pgmap v183: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:19:03.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:02 smithi102 ceph-mon[29216]: pgmap v183: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:19:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:02 smithi161 ceph-mon[32709]: pgmap v183: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:19:05.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:04 smithi183 ceph-mon[33137]: pgmap v184: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:19:05.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:04 smithi102 ceph-mon[29216]: pgmap v184: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:19:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:04 smithi161 ceph-mon[32709]: pgmap v184: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:19:07.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:06 smithi183 ceph-mon[33137]: pgmap v185: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 32 op/s 2024-04-24T23:19:07.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:06 smithi102 ceph-mon[29216]: pgmap v185: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 32 op/s 2024-04-24T23:19:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:06 smithi161 ceph-mon[32709]: pgmap v185: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 32 op/s 2024-04-24T23:19:09.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:08 smithi183 ceph-mon[33137]: pgmap v186: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:19:09.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:08 smithi102 ceph-mon[29216]: pgmap v186: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:19:09.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:08 smithi161 ceph-mon[32709]: pgmap v186: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 40 op/s 2024-04-24T23:19:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:10 smithi183 ceph-mon[33137]: pgmap v187: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:19:11.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:10 smithi102 ceph-mon[29216]: pgmap v187: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:19:11.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:10 smithi161 ceph-mon[32709]: pgmap v187: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:19:12.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:12 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:12.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:11 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:12.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:11 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.525+0000 7ff053458640 1 -- 172.21.15.102:0/1294719839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c101710 msgr2=0x7ff04c10d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.525+0000 7ff053458640 1 --2- 172.21.15.102:0/1294719839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c101710 0x7ff04c10d700 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff044009cd0 tx=0x7ff04402a8e0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.525+0000 7ff053458640 1 -- 172.21.15.102:0/1294719839 shutdown_connections 2024-04-24T23:19:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.525+0000 7ff053458640 1 --2- 172.21.15.102:0/1294719839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c101710 0x7ff04c10d700 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.525+0000 7ff053458640 1 --2- 172.21.15.102:0/1294719839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c100d50 0x7ff04c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.525+0000 7ff053458640 1 --2- 172.21.15.102:0/1294719839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff04c106840 0x7ff04c106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.525+0000 7ff053458640 1 -- 172.21.15.102:0/1294719839 >> 172.21.15.102:0/1294719839 conn(0x7ff04c076bd0 msgr2=0x7ff04c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.525+0000 7ff053458640 1 -- 172.21.15.102:0/1294719839 shutdown_connections 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.525+0000 7ff053458640 1 -- 172.21.15.102:0/1294719839 wait complete. 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff053458640 1 Processor -- start 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff053458640 1 -- start start 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff053458640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c100d50 0x7ff04c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff053458640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff04c101710 0x7ff04c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff053458640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c106840 0x7ff04c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff053458640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c110b00 con 0x7ff04c100d50 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff053458640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c110980 con 0x7ff04c106840 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff053458640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c110c80 con 0x7ff04c101710 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0511f4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c100d50 0x7ff04c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0511f4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c100d50 0x7ff04c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44422/0 (socket says 172.21.15.102:44422) 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0511f4640 1 -- 172.21.15.102:0/3315378309 learned_addr learned my addr 172.21.15.102:0/3315378309 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0509f3640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff04c101710 0x7ff04c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0519f5640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c106840 0x7ff04c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0511f4640 1 -- 172.21.15.102:0/3315378309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff04c101710 msgr2=0x7ff04c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0511f4640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff04c101710 0x7ff04c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0511f4640 1 -- 172.21.15.102:0/3315378309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c106840 msgr2=0x7ff04c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0511f4640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c106840 0x7ff04c19c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0511f4640 1 -- 172.21.15.102:0/3315378309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c19d040 con 0x7ff04c100d50 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.526+0000 7ff0511f4640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c100d50 0x7ff04c198050 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff04800c970 tx=0x7ff04800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.527+0000 7ff0427fc640 1 -- 172.21.15.102:0/3315378309 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048007bb0 con 0x7ff04c100d50 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.527+0000 7ff0427fc640 1 -- 172.21.15.102:0/3315378309 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff048007d50 con 0x7ff04c100d50 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.527+0000 7ff053458640 1 -- 172.21.15.102:0/3315378309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04c19d330 con 0x7ff04c100d50 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.527+0000 7ff0427fc640 1 -- 172.21.15.102:0/3315378309 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04800f450 con 0x7ff04c100d50 2024-04-24T23:19:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.527+0000 7ff053458640 1 -- 172.21.15.102:0/3315378309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff04c1a4bc0 con 0x7ff04c100d50 2024-04-24T23:19:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.529+0000 7ff0427fc640 1 -- 172.21.15.102:0/3315378309 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff048004f20 con 0x7ff04c100d50 2024-04-24T23:19:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff056163740 1 Processor -- start 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff056163740 1 -- start start 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff056163740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 0x55a57e805620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff056163740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e80fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff056163740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e810230 0x55a57e812620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff0427fc640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0380a2da0 0x7ff0380a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff056163740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a57e769cb0 con 0x55a57e805b60 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff056163740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a57e7125b0 con 0x55a57e810230 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff056163740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a57e765c70 con 0x55a57e805240 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff0511f4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e80fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff0519f5640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0380a2da0 0x7ff0380a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff0511f4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e80fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44432/0 (socket says 172.21.15.102:44432) 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff0511f4640 1 -- 172.21.15.102:0/772299853 learned_addr learned my addr 172.21.15.102:0/772299853 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff0519f5640 1 --2- 172.21.15.102:0/772299853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 0x55a57e805620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.530+0000 7ff0509f3640 1 --2- 172.21.15.102:0/772299853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e810230 0x55a57e812620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0427fc640 1 -- 172.21.15.102:0/3315378309 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff0480cf150 con 0x7ff04c100d50 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0511f4640 1 -- 172.21.15.102:0/772299853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 msgr2=0x55a57e805620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0511f4640 1 --2- 172.21.15.102:0/772299853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 0x55a57e805620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0511f4640 1 -- 172.21.15.102:0/772299853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e810230 msgr2=0x55a57e812620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0511f4640 1 --2- 172.21.15.102:0/772299853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e810230 0x55a57e812620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0511f4640 1 -- 172.21.15.102:0/772299853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57e47e210 con 0x55a57e805b60 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0511f4640 1 --2- 172.21.15.102:0/772299853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e80fcf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff04806e6a0 tx=0x7ff04806e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9964f0c6fea00da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0519f5640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0380a2da0 0x7ff0380a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff044005e30 tx=0x7ff044005dc0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0377fe640 1 -- 172.21.15.102:0/772299853 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04801d040 con 0x55a57e805b60 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0377fe640 1 -- 172.21.15.102:0/772299853 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff04809bd60 con 0x55a57e805b60 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.531+0000 7ff0377fe640 1 -- 172.21.15.102:0/772299853 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048097cd0 con 0x55a57e805b60 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 -- 172.21.15.102:0/772299853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 msgr2=0x55a57e80fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 --2- 172.21.15.102:0/772299853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e80fcf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff04806e6a0 tx=0x7ff04806e6d0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 -- 172.21.15.102:0/772299853 shutdown_connections 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 --2- 172.21.15.102:0/772299853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e810230 0x55a57e812620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 --2- 172.21.15.102:0/772299853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e80fcf0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 --2- 172.21.15.102:0/772299853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 0x55a57e805620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 -- 172.21.15.102:0/772299853 >> 172.21.15.102:0/772299853 conn(0x55a57e801560 msgr2=0x55a57e801940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 -- 172.21.15.102:0/772299853 shutdown_connections 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 -- 172.21.15.102:0/772299853 wait complete. 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 -- start start 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.532+0000 7ff056163740 1 Processor -- start 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff056163740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 0x55a57e80d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff056163740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805b60 0x55a57e80d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff056163740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff056163740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a57e813020 con 0x55a57e810230 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff056163740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a57e725b50 con 0x55a57e805b60 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff056163740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a57e8131a0 con 0x55a57e805240 2024-04-24T23:19:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff056163740 10 client.0 _ll_register_callbacks cb 0x55a57e760ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0519f5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0509f3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805b60 0x55a57e80d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0509f3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805b60 0x55a57e80d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:50796/0 (socket says 172.21.15.102:50796) 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0509f3640 1 -- 172.21.15.102:0/4226257586 learned_addr learned my addr 172.21.15.102:0/4226257586 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0511f4640 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 0x55a57e80d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0519f5640 1 -- 172.21.15.102:0/4226257586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 msgr2=0x55a57e80d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0519f5640 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 0x55a57e80d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0519f5640 1 -- 172.21.15.102:0/4226257586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805b60 msgr2=0x55a57e80d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0519f5640 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805b60 0x55a57e80d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.533+0000 7ff0519f5640 1 -- 172.21.15.102:0/4226257586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57e54a620 con 0x55a57e810230 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff0519f5640 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80dbd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff044009a60 tx=0x7ff0440095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab55a3c659957a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff034ff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff034ff9640 1 -- 172.21.15.102:0/4226257586 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04404e030 con 0x55a57e810230 2024-04-24T23:19:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff034ff9640 1 -- 172.21.15.102:0/4226257586 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff044004250 con 0x55a57e810230 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff056163740 1 -- 172.21.15.102:0/4226257586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff04809bd60 con 0x55a57e810230 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff034ff9640 1 -- 172.21.15.102:0/4226257586 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff044004450 con 0x55a57e810230 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff056163740 10 client.15894 fetch_fsmap learned FSMap version 26 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff056163740 1 -- 172.21.15.102:0/4226257586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a57e81f1e0 con 0x55a57e810230 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff034ff9640 1 -- 172.21.15.102:0/4226257586 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff04809bd60 con 0x55a57e810230 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff034ff9640 1 -- 172.21.15.102:0/4226257586 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff044030070 con 0x55a57e810230 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff056163740 10 client.15894 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff056163740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff056163740 20 client.15894 populate_metadata read hostname 'smithi102' 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff056163740 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a57e8206a0 0x55a57e822ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff056163740 4 client.15894 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.534+0000 7ff056163740 1 -- 172.21.15.102:0/4226257586 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a57e7771f0 con 0x55a57e8206a0 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.535+0000 7ff0511f4640 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a57e8206a0 0x55a57e822ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.535+0000 7ff0511f4640 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a57e8206a0 0x55a57e822ad0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=337cb17530dae2ee server_cookie=b8d4bb97497f8b13 in_seq=0 out_seq=0 2024-04-24T23:19:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.535+0000 7ff034ff9640 10 client.15894 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff034ff9640 1 -- 172.21.15.102:0/4226257586 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a57e7771f0 con 0x55a57e8206a0 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff034ff9640 10 client.15894 handle_command_reply: tid=0 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 client.15894 shutdown 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 -- 172.21.15.102:0/4226257586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 msgr2=0x55a57e80dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80dbd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff044009a60 tx=0x7ff0440095c0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 -- 172.21.15.102:0/4226257586 shutdown_connections 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a57e8206a0 0x55a57e822ad0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80dbd0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805b60 0x55a57e80d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 --2- 172.21.15.102:0/4226257586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805240 0x55a57e80d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 -- 172.21.15.102:0/4226257586 >> 172.21.15.102:0/4226257586 conn(0x55a57e801560 msgr2=0x55a57e807220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff034ff9640 0 client.15894 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 -- 172.21.15.102:0/4226257586 shutdown_connections 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 1 -- 172.21.15.102:0/4226257586 wait complete. 2024-04-24T23:19:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.536+0000 7ff056163740 20 client.15894 trim_cache size 0 max 16384 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff056163740 1 Processor -- start 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff056163740 1 -- start start 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff056163740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 0x55a57e80f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff056163740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805b60 0x55a57e81bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff056163740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff056163740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1afc50 con 0x55a57e810230 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff056163740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1ad520 con 0x55a57e805240 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff056163740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1ab390 con 0x55a57e805b60 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0519f5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0519f5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44450/0 (socket says 172.21.15.102:44450) 2024-04-24T23:19:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0519f5640 1 -- 172.21.15.102:0/4040424947 learned_addr learned my addr 172.21.15.102:0/4040424947 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0509f3640 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805b60 0x55a57e81bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0511f4640 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 0x55a57e80f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0519f5640 1 -- 172.21.15.102:0/4040424947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805b60 msgr2=0x55a57e81bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0519f5640 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805b60 0x55a57e81bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0519f5640 1 -- 172.21.15.102:0/4040424947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 msgr2=0x55a57e80f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0519f5640 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 0x55a57e80f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0519f5640 1 -- 172.21.15.102:0/4040424947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57e808d40 con 0x55a57e810230 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0509f3640 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805b60 0x55a57e81bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.554+0000 7ff0519f5640 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80c130 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff044004410 tx=0x7ff044004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584cb030445520b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff036ffd640 1 -- 172.21.15.102:0/4040424947 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04404e040 con 0x55a57e810230 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff036ffd640 1 -- 172.21.15.102:0/4040424947 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff044036070 con 0x55a57e810230 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff036ffd640 1 -- 172.21.15.102:0/4040424947 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044004750 con 0x55a57e810230 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff056163740 1 -- 172.21.15.102:0/4040424947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 msgr2=0x55a57e80c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff056163740 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80c130 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff044004410 tx=0x7ff044004e50 comp rx=0 tx=0).stop 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff056163740 1 -- 172.21.15.102:0/4040424947 shutdown_connections 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff056163740 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e810230 0x55a57e80c130 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff056163740 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e805b60 0x55a57e81bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff056163740 1 --2- 172.21.15.102:0/4040424947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 0x55a57e80f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff056163740 1 -- 172.21.15.102:0/4040424947 >> 172.21.15.102:0/4040424947 conn(0x55a57e801560 msgr2=0x55a57e80ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.555+0000 7ff056163740 1 -- 172.21.15.102:0/4040424947 shutdown_connections 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 1 -- 172.21.15.102:0/4040424947 wait complete. 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 1 -- start start 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 1 Processor -- start 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 0x55a57e8183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e818930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e810230 0x55a57e809440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1ab9e0 con 0x55a57e805b60 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1ace50 con 0x55a57e805240 2024-04-24T23:19:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1ac0b0 con 0x55a57e810230 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff056163740 10 client.0 _ll_register_callbacks cb 0x7ff044004f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff0509f3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e818930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff0509f3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e818930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44460/0 (socket says 172.21.15.102:44460) 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff0509f3640 1 -- 172.21.15.102:0/3470267193 learned_addr learned my addr 172.21.15.102:0/3470267193 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.556+0000 7ff0511f4640 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 0x55a57e8183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff0519f5640 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e810230 0x55a57e809440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff0509f3640 1 -- 172.21.15.102:0/3470267193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e810230 msgr2=0x55a57e809440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff0509f3640 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e810230 0x55a57e809440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff0509f3640 1 -- 172.21.15.102:0/3470267193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 msgr2=0x55a57e8183f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff0509f3640 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 0x55a57e8183f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff0509f3640 1 -- 172.21.15.102:0/3470267193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57e809a30 con 0x55a57e805b60 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff0519f5640 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e810230 0x55a57e809440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff0509f3640 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e818930 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00baa0 tx=0x7ff03c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899cea9b136c3f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff034ff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff034ff9640 1 -- 172.21.15.102:0/3470267193 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c01c670 con 0x55a57e805b60 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff034ff9640 1 -- 172.21.15.102:0/3470267193 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7ff03c01c810 con 0x55a57e805b60 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff056163740 1 -- 172.21.15.102:0/3470267193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff044036070 con 0x55a57e805b60 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff034ff9640 1 -- 172.21.15.102:0/3470267193 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff03c02a050 con 0x55a57e805b60 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff056163740 10 client.15900 fetch_fsmap learned FSMap version 26 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff056163740 1 -- 172.21.15.102:0/3470267193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a57e886570 con 0x55a57e805b60 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.557+0000 7ff034ff9640 1 -- 172.21.15.102:0/3470267193 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff044036070 con 0x55a57e805b60 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff034ff9640 1 -- 172.21.15.102:0/3470267193 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff03c011aa0 con 0x55a57e805b60 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff056163740 10 client.15900 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff056163740 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff056163740 20 client.15900 populate_metadata read hostname 'smithi102' 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff056163740 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a57e886d70 0x55a57e889190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff056163740 4 client.15900 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff056163740 1 -- 172.21.15.102:0/3470267193 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a57e771540 con 0x55a57e886d70 2024-04-24T23:19:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff0511f4640 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a57e886d70 0x55a57e889190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff034ff9640 10 client.15900 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.558+0000 7ff0511f4640 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a57e886d70 0x55a57e889190 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7721663a88e412 server_cookie=5230c73db80facd4 in_seq=0 out_seq=0 2024-04-24T23:19:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff034ff9640 1 -- 172.21.15.102:0/3470267193 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a57e771540 con 0x55a57e886d70 2024-04-24T23:19:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff034ff9640 10 client.15900 handle_command_reply: tid=0 2024-04-24T23:19:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 client.15900 shutdown 2024-04-24T23:19:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 -- 172.21.15.102:0/3470267193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 msgr2=0x55a57e818930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e818930 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00baa0 tx=0x7ff03c00bf60 comp rx=0 tx=0).stop 2024-04-24T23:19:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 -- 172.21.15.102:0/3470267193 shutdown_connections 2024-04-24T23:19:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a57e810230 0x55a57e809440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a57e886d70 0x55a57e889190 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a57e805b60 0x55a57e818930 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 --2- 172.21.15.102:0/3470267193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a57e805240 0x55a57e8183f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 -- 172.21.15.102:0/3470267193 >> 172.21.15.102:0/3470267193 conn(0x55a57e801560 msgr2=0x55a57e806990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff034ff9640 0 client.15900 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 -- 172.21.15.102:0/3470267193 shutdown_connections 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.565+0000 7ff056163740 1 -- 172.21.15.102:0/3470267193 wait complete. 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff056163740 20 client.15900 trim_cache size 0 max 16384 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:19:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:19:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 -- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0380a2da0 msgr2=0x7ff0380a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0380a2da0 0x7ff0380a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff044005e30 tx=0x7ff044005dc0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 -- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c100d50 msgr2=0x7ff04c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c100d50 0x7ff04c198050 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff04800c970 tx=0x7ff04800ce30 comp rx=0 tx=0).stop 2024-04-24T23:19:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 -- 172.21.15.102:0/3315378309 shutdown_connections 2024-04-24T23:19:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04c106840 0x7ff04c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff04c101710 0x7ff04c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0380a2da0 0x7ff0380a5260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 --2- 172.21.15.102:0/3315378309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff04c100d50 0x7ff04c198050 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 -- 172.21.15.102:0/3315378309 >> 172.21.15.102:0/3315378309 conn(0x7ff04c076bd0 msgr2=0x7ff04c100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 -- 172.21.15.102:0/3315378309 shutdown_connections 2024-04-24T23:19:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:12.566+0000 7ff053458640 1 -- 172.21.15.102:0/3315378309 wait complete. 2024-04-24T23:19:13.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:13 smithi183 ceph-mon[33137]: pgmap v188: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:19:13.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:13 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:13.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:13 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:13.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:13 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:19:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:13 smithi102 ceph-mon[29216]: pgmap v188: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:19:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:13 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:13 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:13 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:19:13.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:13 smithi161 ceph-mon[32709]: pgmap v188: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:19:13.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:13 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:13.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:13 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:13.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:13 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:19:15.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:15 smithi183 ceph-mon[33137]: pgmap v189: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 37 MiB/s wr, 41 op/s 2024-04-24T23:19:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:15 smithi102 ceph-mon[29216]: pgmap v189: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 37 MiB/s wr, 41 op/s 2024-04-24T23:19:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:15 smithi161 ceph-mon[32709]: pgmap v189: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 37 MiB/s wr, 41 op/s 2024-04-24T23:19:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:17 smithi183 ceph-mon[33137]: pgmap v190: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:17.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:17 smithi102 ceph-mon[29216]: pgmap v190: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:17.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:17 smithi161 ceph-mon[32709]: pgmap v190: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:19.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:19 smithi183 ceph-mon[33137]: pgmap v191: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:19.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:19 smithi102 ceph-mon[29216]: pgmap v191: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:19 smithi161 ceph-mon[32709]: pgmap v191: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:21.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:21 smithi102 ceph-mon[29216]: pgmap v192: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:21.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:21 smithi161 ceph-mon[32709]: pgmap v192: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:21.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:21 smithi183 ceph-mon[33137]: pgmap v192: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:23.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:23 smithi102 ceph-mon[29216]: pgmap v193: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:23.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:23 smithi161 ceph-mon[32709]: pgmap v193: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:23 smithi183 ceph-mon[33137]: pgmap v193: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:25.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:25 smithi102 ceph-mon[29216]: pgmap v194: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:25.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:25 smithi161 ceph-mon[32709]: pgmap v194: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:25.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:25 smithi183 ceph-mon[33137]: pgmap v194: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 -- 172.21.15.102:0/228268864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 msgr2=0x7fb80c100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 --2- 172.21.15.102:0/228268864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 0x7fb80c100af0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb800009930 tx=0x7fb80002a420 comp rx=0 tx=0).stop 2024-04-24T23:19:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 -- 172.21.15.102:0/228268864 shutdown_connections 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 --2- 172.21.15.102:0/228268864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c101030 0x7fb80c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 --2- 172.21.15.102:0/228268864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 0x7fb80c100af0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 --2- 172.21.15.102:0/228268864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb80c106630 0x7fb80c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 -- 172.21.15.102:0/228268864 >> 172.21.15.102:0/228268864 conn(0x7fb80c0777c0 msgr2=0x7fb80c077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 -- 172.21.15.102:0/228268864 shutdown_connections 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 -- 172.21.15.102:0/228268864 wait complete. 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.660+0000 7fb810ee1640 1 Processor -- start 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb810ee1640 1 -- start start 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb810ee1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 0x7fb80c0fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb810ee1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c101030 0x7fb80c0fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb810ee1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb80c106630 0x7fb80c0fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb810ee1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c110900 con 0x7fb80c100670 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb810ee1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c110780 con 0x7fb80c106630 2024-04-24T23:19:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb810ee1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c110a80 con 0x7fb80c101030 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 0x7fb80c0fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 0x7fb80c0fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56250/0 (socket says 172.21.15.102:56250) 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80a59c640 1 -- 172.21.15.102:0/2042986477 learned_addr learned my addr 172.21.15.102:0/2042986477 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb80c106630 0x7fb80c0fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb809d9b640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c101030 0x7fb80c0fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80a59c640 1 -- 172.21.15.102:0/2042986477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c101030 msgr2=0x7fb80c0fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80a59c640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c101030 0x7fb80c0fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80a59c640 1 -- 172.21.15.102:0/2042986477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb80c106630 msgr2=0x7fb80c0fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80a59c640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb80c106630 0x7fb80c0fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80a59c640 1 -- 172.21.15.102:0/2042986477 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c1a6460 con 0x7fb80c100670 2024-04-24T23:19:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb80a59c640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 0x7fb80c0fd600 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00d950 tx=0x7fb7fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb7fb7fe640 1 -- 172.21.15.102:0/2042986477 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc017030 con 0x7fb80c100670 2024-04-24T23:19:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb810ee1640 1 -- 172.21.15.102:0/2042986477 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c1a6750 con 0x7fb80c100670 2024-04-24T23:19:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.661+0000 7fb7fb7fe640 1 -- 172.21.15.102:0/2042986477 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7fc004020 con 0x7fb80c100670 2024-04-24T23:19:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.662+0000 7fb7fb7fe640 1 -- 172.21.15.102:0/2042986477 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0047a0 con 0x7fb80c100670 2024-04-24T23:19:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.662+0000 7fb810ee1640 1 -- 172.21.15.102:0/2042986477 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c1a6cc0 con 0x7fb80c100670 2024-04-24T23:19:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.664+0000 7fb7fb7fe640 1 -- 172.21.15.102:0/2042986477 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb7fc0041d0 con 0x7fb80c100670 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.664+0000 7fb813bf3740 1 Processor -- start 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.664+0000 7fb7fb7fe640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb7f40a2da0 0x7fb7f40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.664+0000 7fb813bf3740 1 -- start start 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa4c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa56cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa59620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb813bf3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556aef9b0cb0 con 0x556aefa4cb60 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb813bf3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556aef9595b0 con 0x556aefa57230 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb813bf3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556aef9acc70 con 0x556aefa4c240 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb7f40a2da0 0x7fb7f40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb7fb7fe640 1 -- 172.21.15.102:0/2042986477 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb7fc0ceef0 con 0x7fb80c100670 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb809d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa56cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb809d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa56cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56266/0 (socket says 172.21.15.102:56266) 2024-04-24T23:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb809d9b640 1 -- 172.21.15.102:0/668968470 learned_addr learned my addr 172.21.15.102:0/668968470 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa59620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb80a59c640 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa4c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb809d9b640 1 -- 172.21.15.102:0/668968470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 msgr2=0x556aefa4c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb809d9b640 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa4c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb809d9b640 1 -- 172.21.15.102:0/668968470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 msgr2=0x556aefa59620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb809d9b640 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa59620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb809d9b640 1 -- 172.21.15.102:0/668968470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aef6c5210 con 0x556aefa4cb60 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb7f40a2da0 0x7fb7f40a5260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb80400ad70 tx=0x7fb80400e720 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb80a59c640 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa4c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb809d9b640 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa56cf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb80002aa10 tx=0x7fb800004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d15c4078b9be6bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.665+0000 7fb7f8ff9640 1 -- 172.21.15.102:0/668968470 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800035030 con 0x556aefa4cb60 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.666+0000 7fb7f8ff9640 1 -- 172.21.15.102:0/668968470 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8000046c0 con 0x556aefa4cb60 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.666+0000 7fb7f8ff9640 1 -- 172.21.15.102:0/668968470 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800005d50 con 0x556aefa4cb60 2024-04-24T23:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.666+0000 7fb813bf3740 1 -- 172.21.15.102:0/668968470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 msgr2=0x556aefa56cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.666+0000 7fb813bf3740 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa56cf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb80002aa10 tx=0x7fb800004390 comp rx=0 tx=0).stop 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.666+0000 7fb813bf3740 1 -- 172.21.15.102:0/668968470 shutdown_connections 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.666+0000 7fb813bf3740 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa59620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.666+0000 7fb813bf3740 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa56cf0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.666+0000 7fb813bf3740 1 --2- 172.21.15.102:0/668968470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa4c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.666+0000 7fb813bf3740 1 -- 172.21.15.102:0/668968470 >> 172.21.15.102:0/668968470 conn(0x556aefa48560 msgr2=0x556aefa48940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 -- 172.21.15.102:0/668968470 shutdown_connections 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 -- 172.21.15.102:0/668968470 wait complete. 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 -- start start 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 Processor -- start 2024-04-24T23:19:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa54130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa54670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa54bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556aefa59fc0 con 0x556aefa4cb60 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556aef96cb50 con 0x556aefa57230 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556aefa5a140 con 0x556aefa4c240 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb813bf3740 10 client.0 _ll_register_callbacks cb 0x556aef9a7ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.667+0000 7fb809d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa54670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb809d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa54670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56270/0 (socket says 172.21.15.102:56270) 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb809d9b640 1 -- 172.21.15.102:0/3295618191 learned_addr learned my addr 172.21.15.102:0/3295618191 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa54bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb80a59c640 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa54130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb809d9b640 1 -- 172.21.15.102:0/3295618191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 msgr2=0x556aefa54130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb809d9b640 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa54130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb809d9b640 1 -- 172.21.15.102:0/3295618191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 msgr2=0x556aefa54bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb809d9b640 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa54bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb809d9b640 1 -- 172.21.15.102:0/3295618191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aef791620 con 0x556aefa4cb60 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb809d9b640 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa54670 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb800006010 tx=0x7fb800005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788bfd6a7b26842e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb7f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/3295618191 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800035030 con 0x556aefa4cb60 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/3295618191 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb8000314a0 con 0x556aefa4cb60 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb813bf3740 1 -- 172.21.15.102:0/3295618191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8000046c0 con 0x556aefa4cb60 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/3295618191 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb800005d50 con 0x556aefa4cb60 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb813bf3740 10 client.15912 fetch_fsmap learned FSMap version 26 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb813bf3740 1 -- 172.21.15.102:0/3295618191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556aefa660d0 con 0x556aefa4cb60 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.668+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/3295618191 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8000046c0 con 0x556aefa4cb60 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/3295618191 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb80003b690 con 0x556aefa4cb60 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb813bf3740 10 client.15912 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:19:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb813bf3740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb813bf3740 20 client.15912 populate_metadata read hostname 'smithi102' 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb813bf3740 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x556aefa67770 0x556aefa69bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb813bf3740 4 client.15912 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb813bf3740 1 -- 172.21.15.102:0/3295618191 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aef9be1f0 con 0x556aefa67770 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb80a59c640 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x556aefa67770 0x556aefa69bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb7f1ffb640 10 client.15912 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.669+0000 7fb80a59c640 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x556aefa67770 0x556aefa69bd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732706ae41b88f7 server_cookie=1e126b691e4a697f in_seq=0 out_seq=0 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/3295618191 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556aef9be1f0 con 0x556aefa67770 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb7f1ffb640 10 client.15912 handle_command_reply: tid=0 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 client.15912 shutdown 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 -- 172.21.15.102:0/3295618191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 msgr2=0x556aefa54670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa54670 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb800006010 tx=0x7fb800005230 comp rx=0 tx=0).stop 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 -- 172.21.15.102:0/3295618191 shutdown_connections 2024-04-24T23:19:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa54bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x556aefa67770 0x556aefa69bd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4cb60 0x556aefa54670 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 --2- 172.21.15.102:0/3295618191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa54130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb7f1ffb640 0 client.15912 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 -- 172.21.15.102:0/3295618191 >> 172.21.15.102:0/3295618191 conn(0x556aefa48560 msgr2=0x556aefa4e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.670+0000 7fb813bf3740 1 -- 172.21.15.102:0/3295618191 shutdown_connections 2024-04-24T23:19:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.671+0000 7fb813bf3740 1 -- 172.21.15.102:0/3295618191 wait complete. 2024-04-24T23:19:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.671+0000 7fb813bf3740 20 client.15912 trim_cache size 0 max 16384 2024-04-24T23:19:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb813bf3740 1 Processor -- start 2024-04-24T23:19:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb813bf3740 1 -- start start 2024-04-24T23:19:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4c240 0x556aefa62eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4cb60 0x556aefa53110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa6f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb813bf3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1b1dc0 con 0x556aefa4c240 2024-04-24T23:19:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb813bf3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1af690 con 0x556aefa57230 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb813bf3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ad500 con 0x556aefa4cb60 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb80a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4c240 0x556aefa62eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb80a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4c240 0x556aefa62eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56278/0 (socket says 172.21.15.102:56278) 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb80a59c640 1 -- 172.21.15.102:0/1258551996 learned_addr learned my addr 172.21.15.102:0/1258551996 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa6f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb809d9b640 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4cb60 0x556aefa53110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb80a59c640 1 -- 172.21.15.102:0/1258551996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4cb60 msgr2=0x556aefa53110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb80a59c640 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4cb60 0x556aefa53110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb80a59c640 1 -- 172.21.15.102:0/1258551996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 msgr2=0x556aefa6f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb80a59c640 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa6f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb80a59c640 1 -- 172.21.15.102:0/1258551996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aefa57060 con 0x556aefa4c240 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.688+0000 7fb809d9b640 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4cb60 0x556aefa53110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.689+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa6f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.689+0000 7fb80a59c640 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4c240 0x556aefa62eb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc09b100 tx=0x7fb7fc0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393bca3c23c628ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.689+0000 7fb7f3fff640 1 -- 172.21.15.102:0/1258551996 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc09f7b0 con 0x556aefa4c240 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.689+0000 7fb7f3fff640 1 -- 172.21.15.102:0/1258551996 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7fc014040 con 0x556aefa4c240 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.689+0000 7fb7f3fff640 1 -- 172.21.15.102:0/1258551996 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0b4270 con 0x556aefa4c240 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 -- 172.21.15.102:0/1258551996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4c240 msgr2=0x556aefa62eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4c240 0x556aefa62eb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc09b100 tx=0x7fb7fc0bb040 comp rx=0 tx=0).stop 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 -- 172.21.15.102:0/1258551996 shutdown_connections 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa57230 0x556aefa6f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4cb60 0x556aefa53110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 --2- 172.21.15.102:0/1258551996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa4c240 0x556aefa62eb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 -- 172.21.15.102:0/1258551996 >> 172.21.15.102:0/1258551996 conn(0x556aefa48560 msgr2=0x556aefa52a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 -- 172.21.15.102:0/1258551996 shutdown_connections 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 -- 172.21.15.102:0/1258551996 wait complete. 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 -- start start 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 Processor -- start 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa5f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa4cb60 0x556aefa5f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa57230 0x556aefa503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1adb50 con 0x556aefa57230 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1aefc0 con 0x556aefa4cb60 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ae220 con 0x556aefa4c240 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb813bf3740 10 client.0 _ll_register_callbacks cb 0x7fb800004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb80ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa57230 0x556aefa503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb80ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa57230 0x556aefa503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56294/0 (socket says 172.21.15.102:56294) 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb80ad9d640 1 -- 172.21.15.102:0/122369825 learned_addr learned my addr 172.21.15.102:0/122369825 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb809d9b640 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa4cb60 0x556aefa5f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.690+0000 7fb80a59c640 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa5f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb80ad9d640 1 -- 172.21.15.102:0/122369825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 msgr2=0x556aefa5f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa5f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb80ad9d640 1 -- 172.21.15.102:0/122369825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa4cb60 msgr2=0x556aefa5f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa4cb60 0x556aefa5f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb80ad9d640 1 -- 172.21.15.102:0/122369825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aefa50990 con 0x556aefa57230 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb80ad9d640 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa57230 0x556aefa503a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb804020e80 tx=0x7fb804004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d612e8270e467c59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb7f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/122369825 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804012e60 con 0x556aefa57230 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/122369825 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fb804011440 con 0x556aefa57230 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb813bf3740 1 -- 172.21.15.102:0/122369825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7fc014040 con 0x556aefa57230 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/122369825 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb804011680 con 0x556aefa57230 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb813bf3740 10 client.15918 fetch_fsmap learned FSMap version 26 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb813bf3740 1 -- 172.21.15.102:0/122369825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556aefacd590 con 0x556aefa57230 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.691+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/122369825 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7fc014040 con 0x556aefa57230 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/122369825 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb804027070 con 0x556aefa57230 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb813bf3740 10 client.15918 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb813bf3740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb813bf3740 20 client.15918 populate_metadata read hostname 'smithi102' 2024-04-24T23:19:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb813bf3740 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x556aefacdd90 0x556aefad01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb813bf3740 4 client.15918 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:19:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb813bf3740 1 -- 172.21.15.102:0/122369825 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556aef9b8540 con 0x556aefacdd90 2024-04-24T23:19:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb80a59c640 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x556aefacdd90 0x556aefad01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb80a59c640 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x556aefacdd90 0x556aefad01b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4ce7d188dda2ed server_cookie=fd473c532a7a0093 in_seq=0 out_seq=0 2024-04-24T23:19:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.692+0000 7fb7f1ffb640 10 client.15918 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb7f1ffb640 1 -- 172.21.15.102:0/122369825 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556aef9b8540 con 0x556aefacdd90 2024-04-24T23:19:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb7f1ffb640 10 client.15918 handle_command_reply: tid=0 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 client.15918 shutdown 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 -- 172.21.15.102:0/122369825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa57230 msgr2=0x556aefa503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa57230 0x556aefa503a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb804020e80 tx=0x7fb804004ab0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 -- 172.21.15.102:0/122369825 shutdown_connections 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x556aefacdd90 0x556aefad01b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556aefa57230 0x556aefa503a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556aefa4cb60 0x556aefa5f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 --2- 172.21.15.102:0/122369825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aefa4c240 0x556aefa5f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 -- 172.21.15.102:0/122369825 >> 172.21.15.102:0/122369825 conn(0x556aefa48560 msgr2=0x556aefa540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb7f1ffb640 0 client.15918 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 -- 172.21.15.102:0/122369825 shutdown_connections 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 1 -- 172.21.15.102:0/122369825 wait complete. 2024-04-24T23:19:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.701+0000 7fb813bf3740 20 client.15918 trim_cache size 0 max 16384 2024-04-24T23:19:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:19:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:19:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:19:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 -- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb7f40a2da0 msgr2=0x7fb7f40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb7f40a2da0 0x7fb7f40a5260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb80400ad70 tx=0x7fb80400e720 comp rx=0 tx=0).stop 2024-04-24T23:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 -- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 msgr2=0x7fb80c0fd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 0x7fb80c0fd600 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00d950 tx=0x7fb7fc00de10 comp rx=0 tx=0).stop 2024-04-24T23:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 -- 172.21.15.102:0/2042986477 shutdown_connections 2024-04-24T23:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb80c106630 0x7fb80c0fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c101030 0x7fb80c0fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb7f40a2da0 0x7fb7f40a5260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 --2- 172.21.15.102:0/2042986477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb80c100670 0x7fb80c0fd600 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 -- 172.21.15.102:0/2042986477 >> 172.21.15.102:0/2042986477 conn(0x7fb80c0777c0 msgr2=0x7fb80c069130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 -- 172.21.15.102:0/2042986477 shutdown_connections 2024-04-24T23:19:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:25.702+0000 7fb810ee1640 1 -- 172.21.15.102:0/2042986477 wait complete. 2024-04-24T23:19:27.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:27 smithi102 ceph-mon[29216]: pgmap v195: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:19:27.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:27 smithi161 ceph-mon[32709]: pgmap v195: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:19:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:27 smithi183 ceph-mon[33137]: pgmap v195: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:19:29.315 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:29 smithi183 ceph-mon[33137]: pgmap v196: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:29.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:29 smithi102 ceph-mon[29216]: pgmap v196: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:29.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:29 smithi161 ceph-mon[32709]: pgmap v196: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:31.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:31 smithi102 ceph-mon[29216]: pgmap v197: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:31.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:31 smithi161 ceph-mon[32709]: pgmap v197: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:31 smithi183 ceph-mon[33137]: pgmap v197: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:33.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:33 smithi102 ceph-mon[29216]: pgmap v198: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:33.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:33 smithi161 ceph-mon[32709]: pgmap v198: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:33.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:33 smithi183 ceph-mon[33137]: pgmap v198: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:35.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:35 smithi102 ceph-mon[29216]: pgmap v199: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:19:35.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:35 smithi161 ceph-mon[32709]: pgmap v199: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:19:35.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:35 smithi183 ceph-mon[33137]: pgmap v199: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:19:37.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:37 smithi102 ceph-mon[29216]: pgmap v200: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-04-24T23:19:37.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:37 smithi161 ceph-mon[32709]: pgmap v200: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-04-24T23:19:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:37 smithi183 ceph-mon[33137]: pgmap v200: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 19 op/s 2024-04-24T23:19:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.795+0000 7fec70a64640 1 -- 172.21.15.102:0/3822730474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c10bd10 msgr2=0x7fec6c10e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.795+0000 7fec70a64640 1 --2- 172.21.15.102:0/3822730474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c10bd10 0x7fec6c10e1b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fec64009ca0 tx=0x7fec6402a9e0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.795+0000 7fec70a64640 1 -- 172.21.15.102:0/3822730474 shutdown_connections 2024-04-24T23:19:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.795+0000 7fec70a64640 1 --2- 172.21.15.102:0/3822730474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c10bd10 0x7fec6c10e1b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.795+0000 7fec70a64640 1 --2- 172.21.15.102:0/3822730474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec6c1032f0 0x7fec6c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.795+0000 7fec70a64640 1 --2- 172.21.15.102:0/3822730474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec6c1029d0 0x7fec6c102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.795+0000 7fec70a64640 1 -- 172.21.15.102:0/3822730474 >> 172.21.15.102:0/3822730474 conn(0x7fec6c0777c0 msgr2=0x7fec6c077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.796+0000 7fec70a64640 1 -- 172.21.15.102:0/3822730474 shutdown_connections 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.796+0000 7fec70a64640 1 -- 172.21.15.102:0/3822730474 wait complete. 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.796+0000 7fec70a64640 1 Processor -- start 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.796+0000 7fec70a64640 1 -- start start 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec70a64640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec6c1029d0 0x7fec6c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec70a64640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c1032f0 0x7fec6c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec70a64640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec6c10bd10 0x7fec6c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec70a64640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c111a10 con 0x7fec6c1032f0 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec70a64640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c111890 con 0x7fec6c1029d0 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec70a64640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c111b90 con 0x7fec6c10bd10 2024-04-24T23:19:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec69d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c1032f0 0x7fec6c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec69d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c1032f0 0x7fec6c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36794/0 (socket says 172.21.15.102:36794) 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec69d9b640 1 -- 172.21.15.102:0/1875232618 learned_addr learned my addr 172.21.15.102:0/1875232618 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec6a59c640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec6c1029d0 0x7fec6c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec6c10bd10 0x7fec6c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec69d9b640 1 -- 172.21.15.102:0/1875232618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec6c10bd10 msgr2=0x7fec6c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec69d9b640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec6c10bd10 0x7fec6c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec69d9b640 1 -- 172.21.15.102:0/1875232618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec6c1029d0 msgr2=0x7fec6c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec69d9b640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec6c1029d0 0x7fec6c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec69d9b640 1 -- 172.21.15.102:0/1875232618 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c19ce20 con 0x7fec6c1032f0 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec69d9b640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c1032f0 0x7fec6c198370 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fec6000d950 tx=0x7fec6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec5b7fe640 1 -- 172.21.15.102:0/1875232618 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60017030 con 0x7fec6c1032f0 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec5b7fe640 1 -- 172.21.15.102:0/1875232618 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec60004020 con 0x7fec6c1032f0 2024-04-24T23:19:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec70a64640 1 -- 172.21.15.102:0/1875232618 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6c19d110 con 0x7fec6c1032f0 2024-04-24T23:19:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec5b7fe640 1 -- 172.21.15.102:0/1875232618 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec600047a0 con 0x7fec6c1032f0 2024-04-24T23:19:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.797+0000 7fec70a64640 1 -- 172.21.15.102:0/1875232618 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6c1a49a0 con 0x7fec6c1032f0 2024-04-24T23:19:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.800+0000 7fec5b7fe640 1 -- 172.21.15.102:0/1875232618 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fec600041d0 con 0x7fec6c1032f0 2024-04-24T23:19:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec73775740 1 Processor -- start 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec73775740 1 -- start start 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec73775740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db02620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec73775740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db0ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec73775740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db0f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec73775740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55885da66cb0 con 0x55885db02240 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec73775740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55885da0f5b0 con 0x55885db0d230 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec73775740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55885da62c70 con 0x55885db02b60 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec5b7fe640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fec540a2e20 0x7fec540a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec6ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db02620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec6ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db02620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36800/0 (socket says 172.21.15.102:36800) 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec6ad9d640 1 -- 172.21.15.102:0/1470993585 learned_addr learned my addr 172.21.15.102:0/1470993585 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec6a59c640 1 --2- 172.21.15.102:0/1470993585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db0ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fec540a2e20 0x7fec540a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec69d9b640 1 --2- 172.21.15.102:0/1470993585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db0f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.801+0000 7fec5b7fe640 1 -- 172.21.15.102:0/1875232618 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fec600cee10 con 0x7fec6c1032f0 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec6ad9d640 1 -- 172.21.15.102:0/1470993585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 msgr2=0x55885db0ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/1470993585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db0ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec6ad9d640 1 -- 172.21.15.102:0/1470993585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 msgr2=0x55885db0f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/1470993585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db0f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec6ad9d640 1 -- 172.21.15.102:0/1470993585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55885d77b210 con 0x55885db02240 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/1470993585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db02620 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fec64035000 tx=0x7fec6402e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6809cd6f16f6e20c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fec540a2e20 0x7fec540a52e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec6402e950 tx=0x7fec64048000 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec58ff9640 1 -- 172.21.15.102:0/1470993585 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64053070 con 0x55885db02240 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec58ff9640 1 -- 172.21.15.102:0/1470993585 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec64003970 con 0x55885db02240 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.802+0000 7fec58ff9640 1 -- 172.21.15.102:0/1470993585 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64041040 con 0x55885db02240 2024-04-24T23:19:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 -- 172.21.15.102:0/1470993585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 msgr2=0x55885db02620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 --2- 172.21.15.102:0/1470993585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db02620 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fec64035000 tx=0x7fec6402e8e0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 -- 172.21.15.102:0/1470993585 shutdown_connections 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 --2- 172.21.15.102:0/1470993585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db0f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 --2- 172.21.15.102:0/1470993585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db0ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 --2- 172.21.15.102:0/1470993585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db02620 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 -- 172.21.15.102:0/1470993585 >> 172.21.15.102:0/1470993585 conn(0x55885dafe560 msgr2=0x55885dafe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 -- 172.21.15.102:0/1470993585 shutdown_connections 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 -- 172.21.15.102:0/1470993585 wait complete. 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 -- start start 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.803+0000 7fec73775740 1 Processor -- start 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec73775740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db02240 0x55885db0a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec73775740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec73775740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db0d230 0x55885db0ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec73775740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55885db0ffc0 con 0x55885db0d230 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec73775740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55885da22b50 con 0x55885db02240 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec73775740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55885db10140 con 0x55885db02b60 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec73775740 10 client.0 _ll_register_callbacks cb 0x55885da5dca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db0d230 0x55885db0ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db0d230 0x55885db0ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36804/0 (socket says 172.21.15.102:36804) 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6ad9d640 1 -- 172.21.15.102:0/2592270599 learned_addr learned my addr 172.21.15.102:0/2592270599 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6a59c640 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db02240 0x55885db0a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec69d9b640 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6ad9d640 1 -- 172.21.15.102:0/2592270599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 msgr2=0x55885db0a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db0a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6ad9d640 1 -- 172.21.15.102:0/2592270599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db02240 msgr2=0x55885db0a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db02240 0x55885db0a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6ad9d640 1 -- 172.21.15.102:0/2592270599 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55885d847620 con 0x55885db0d230 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db0d230 0x55885db0ac90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fec6402afd0 tx=0x7fec6402e550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320bdee48a7e122f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.804+0000 7fec51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec51ffb640 1 -- 172.21.15.102:0/2592270599 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6405c020 con 0x55885db0d230 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec51ffb640 1 -- 172.21.15.102:0/2592270599 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fec64003b20 con 0x55885db0d230 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec73775740 1 -- 172.21.15.102:0/2592270599 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec64003970 con 0x55885db0d230 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec51ffb640 1 -- 172.21.15.102:0/2592270599 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec64053040 con 0x55885db0d230 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec73775740 10 client.15930 fetch_fsmap learned FSMap version 26 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec73775740 1 -- 172.21.15.102:0/2592270599 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55885db1c210 con 0x55885db0d230 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec51ffb640 1 -- 172.21.15.102:0/2592270599 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec64003970 con 0x55885db0d230 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec51ffb640 1 -- 172.21.15.102:0/2592270599 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fec6404d030 con 0x55885db0d230 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec73775740 10 client.15930 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec73775740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec73775740 20 client.15930 populate_metadata read hostname 'smithi102' 2024-04-24T23:19:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec73775740 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55885db1d560 0x55885db1f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec73775740 4 client.15930 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.805+0000 7fec73775740 1 -- 172.21.15.102:0/2592270599 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55885da741f0 con 0x55885db1d560 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.806+0000 7fec6a59c640 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55885db1d560 0x55885db1f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.806+0000 7fec51ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.806+0000 7fec6a59c640 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55885db1d560 0x55885db1f950 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68c612fda549e6 server_cookie=c2497c9e44063773 in_seq=0 out_seq=0 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec51ffb640 1 -- 172.21.15.102:0/2592270599 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55885da741f0 con 0x55885db1d560 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec51ffb640 10 client.15930 handle_command_reply: tid=0 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 client.15930 shutdown 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 -- 172.21.15.102:0/2592270599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db0d230 msgr2=0x55885db0ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db0d230 0x55885db0ac90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fec6402afd0 tx=0x7fec6402e550 comp rx=0 tx=0).stop 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 -- 172.21.15.102:0/2592270599 shutdown_connections 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55885db1d560 0x55885db1f950 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db0d230 0x55885db0ac90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec51ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 --2- 172.21.15.102:0/2592270599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db02240 0x55885db0a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 -- 172.21.15.102:0/2592270599 >> 172.21.15.102:0/2592270599 conn(0x55885dafe560 msgr2=0x55885db04360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 -- 172.21.15.102:0/2592270599 shutdown_connections 2024-04-24T23:19:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 1 -- 172.21.15.102:0/2592270599 wait complete. 2024-04-24T23:19:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.807+0000 7fec73775740 20 client.15930 trim_cache size 0 max 16384 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.825+0000 7fec73775740 1 Processor -- start 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec73775740 1 -- start start 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec73775740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db0c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec73775740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db04ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec73775740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db22970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec73775740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1afca0 con 0x55885db02240 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec73775740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1ad570 con 0x55885db0d230 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec73775740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1ab3e0 con 0x55885db02b60 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db0c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6a59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db0c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36820/0 (socket says 172.21.15.102:36820) 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6a59c640 1 -- 172.21.15.102:0/2473225762 learned_addr learned my addr 172.21.15.102:0/2473225762 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db22970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6a59c640 1 -- 172.21.15.102:0/2473225762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 msgr2=0x55885db04ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec69d9b640 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db04ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6a59c640 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db04ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6a59c640 1 -- 172.21.15.102:0/2473225762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 msgr2=0x55885db22970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6a59c640 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db22970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6a59c640 1 -- 172.21.15.102:0/2473225762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55885db04ff0 con 0x55885db02240 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6ad9d640 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db22970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.826+0000 7fec6a59c640 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db0c940 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55885db0fd20 tx=0x7fec5c00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bebfd6114e1ea0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec53fff640 1 -- 172.21.15.102:0/2473225762 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004220 con 0x55885db02240 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec53fff640 1 -- 172.21.15.102:0/2473225762 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec5c024030 con 0x55885db02240 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec53fff640 1 -- 172.21.15.102:0/2473225762 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0039c0 con 0x55885db02240 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec73775740 1 -- 172.21.15.102:0/2473225762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 msgr2=0x55885db0c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec73775740 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db0c940 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55885db0fd20 tx=0x7fec5c00a2d0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec73775740 1 -- 172.21.15.102:0/2473225762 shutdown_connections 2024-04-24T23:19:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec73775740 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db22970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec73775740 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02b60 0x55885db04ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec73775740 1 --2- 172.21.15.102:0/2473225762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02240 0x55885db0c940 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.827+0000 7fec73775740 1 -- 172.21.15.102:0/2473225762 >> 172.21.15.102:0/2473225762 conn(0x55885dafe560 msgr2=0x55885db09240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 -- 172.21.15.102:0/2473225762 shutdown_connections 2024-04-24T23:19:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 -- 172.21.15.102:0/2473225762 wait complete. 2024-04-24T23:19:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 -- start start 2024-04-24T23:19:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 Processor -- start 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02240 0x55885db20b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02b60 0x55885db21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db82740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1aba30 con 0x55885db02b60 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1acea0 con 0x55885db0d230 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1ac100 con 0x55885db02240 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec73775740 10 client.0 _ll_register_callbacks cb 0x7fec64051070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec69d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02b60 0x55885db21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec69d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02b60 0x55885db21090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36836/0 (socket says 172.21.15.102:36836) 2024-04-24T23:19:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.828+0000 7fec69d9b640 1 -- 172.21.15.102:0/1118288603 learned_addr learned my addr 172.21.15.102:0/1118288603 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec6a59c640 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02240 0x55885db20b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec69d9b640 1 -- 172.21.15.102:0/1118288603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02240 msgr2=0x55885db20b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec69d9b640 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02240 0x55885db20b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec69d9b640 1 -- 172.21.15.102:0/1118288603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 msgr2=0x55885db82740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec69d9b640 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db82740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec69d9b640 1 -- 172.21.15.102:0/1118288603 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55885db82c80 con 0x55885db02b60 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec69d9b640 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02b60 0x55885db21090 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fec600cc5b0 tx=0x7fec60018f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f954a53c664034 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec51ffb640 1 -- 172.21.15.102:0/1118288603 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6009ac80 con 0x55885db02b60 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec51ffb640 1 -- 172.21.15.102:0/1118288603 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7fec60021cc0 con 0x55885db02b60 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec73775740 1 -- 172.21.15.102:0/1118288603 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5c024030 con 0x55885db02b60 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec51ffb640 1 -- 172.21.15.102:0/1118288603 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec6009ba20 con 0x55885db02b60 2024-04-24T23:19:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec73775740 10 client.15936 fetch_fsmap learned FSMap version 26 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec73775740 1 -- 172.21.15.102:0/1118288603 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55885db83f90 con 0x55885db02b60 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec51ffb640 1 -- 172.21.15.102:0/1118288603 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec5c024030 con 0x55885db02b60 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec51ffb640 1 -- 172.21.15.102:0/1118288603 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fec60009a30 con 0x55885db02b60 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec73775740 10 client.15936 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec73775740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec73775740 20 client.15936 populate_metadata read hostname 'smithi102' 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.829+0000 7fec73775740 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55885db846f0 0x55885db86b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.830+0000 7fec73775740 4 client.15936 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.830+0000 7fec73775740 1 -- 172.21.15.102:0/1118288603 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55885da6e540 con 0x55885db846f0 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.830+0000 7fec6a59c640 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55885db846f0 0x55885db86b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.830+0000 7fec51ffb640 10 client.15936 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.830+0000 7fec6a59c640 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55885db846f0 0x55885db86b10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1967ca8a74a5c73f server_cookie=1b5b1312031616ea in_seq=0 out_seq=0 2024-04-24T23:19:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec51ffb640 1 -- 172.21.15.102:0/1118288603 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55885da6e540 con 0x55885db846f0 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec51ffb640 10 client.15936 handle_command_reply: tid=0 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 client.15936 shutdown 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 -- 172.21.15.102:0/1118288603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02b60 msgr2=0x55885db21090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02b60 0x55885db21090 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fec600cc5b0 tx=0x7fec60018f10 comp rx=0 tx=0).stop 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 -- 172.21.15.102:0/1118288603 shutdown_connections 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55885db0d230 0x55885db82740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55885db846f0 0x55885db86b10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55885db02b60 0x55885db21090 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 --2- 172.21.15.102:0/1118288603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55885db02240 0x55885db20b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec51ffb640 0 client.15936 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 -- 172.21.15.102:0/1118288603 >> 172.21.15.102:0/1118288603 conn(0x55885dafe560 msgr2=0x55885db089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 -- 172.21.15.102:0/1118288603 shutdown_connections 2024-04-24T23:19:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 1 -- 172.21.15.102:0/1118288603 wait complete. 2024-04-24T23:19:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.847+0000 7fec73775740 20 client.15936 trim_cache size 0 max 16384 2024-04-24T23:19:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:19:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:19:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:19:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:19:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 -- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fec540a2e20 msgr2=0x7fec540a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fec540a2e20 0x7fec540a52e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec6402e950 tx=0x7fec64048000 comp rx=0 tx=0).stop 2024-04-24T23:19:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 -- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c1032f0 msgr2=0x7fec6c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c1032f0 0x7fec6c198370 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fec6000d950 tx=0x7fec6000de10 comp rx=0 tx=0).stop 2024-04-24T23:19:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 -- 172.21.15.102:0/1875232618 shutdown_connections 2024-04-24T23:19:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec6c10bd10 0x7fec6c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fec540a2e20 0x7fec540a52e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec6c1032f0 0x7fec6c198370 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 --2- 172.21.15.102:0/1875232618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec6c1029d0 0x7fec6c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 -- 172.21.15.102:0/1875232618 >> 172.21.15.102:0/1875232618 conn(0x7fec6c0777c0 msgr2=0x7fec6c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 -- 172.21.15.102:0/1875232618 shutdown_connections 2024-04-24T23:19:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:38.848+0000 7fec70a64640 1 -- 172.21.15.102:0/1875232618 wait complete. 2024-04-24T23:19:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:39 smithi102 ceph-mon[29216]: pgmap v201: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 27 op/s 2024-04-24T23:19:39.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:39 smithi161 ceph-mon[32709]: pgmap v201: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 27 op/s 2024-04-24T23:19:39.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:39 smithi183 ceph-mon[33137]: pgmap v201: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 27 op/s 2024-04-24T23:19:41.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:41 smithi102 ceph-mon[29216]: pgmap v202: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:41.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:41 smithi161 ceph-mon[32709]: pgmap v202: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:41.514 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:41 smithi183 ceph-mon[33137]: pgmap v202: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:43.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:43 smithi102 ceph-mon[29216]: pgmap v203: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:43.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:43 smithi161 ceph-mon[32709]: pgmap v203: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:43.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:43 smithi183 ceph-mon[33137]: pgmap v203: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:45.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:45 smithi102 ceph-mon[29216]: pgmap v204: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:19:45.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:45 smithi161 ceph-mon[32709]: pgmap v204: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:19:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:45 smithi183 ceph-mon[33137]: pgmap v204: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:19:47.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:47 smithi102 ceph-mon[29216]: pgmap v205: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:47.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:47 smithi161 ceph-mon[32709]: pgmap v205: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:47 smithi183 ceph-mon[33137]: pgmap v205: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:49.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:49 smithi102 ceph-mon[29216]: pgmap v206: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:49.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:49 smithi161 ceph-mon[32709]: pgmap v206: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:49 smithi183 ceph-mon[33137]: pgmap v206: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:51.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:51 smithi102 ceph-mon[29216]: pgmap v207: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:51.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:51 smithi161 ceph-mon[32709]: pgmap v207: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:51.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:51 smithi183 ceph-mon[33137]: pgmap v207: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.947+0000 7f63ce872640 1 -- 172.21.15.102:0/810768282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8106630 msgr2=0x7f63c8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.947+0000 7f63ce872640 1 --2- 172.21.15.102:0/810768282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8106630 0x7f63c8106a10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009930 tx=0x7f63b802a420 comp rx=0 tx=0).stop 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.947+0000 7f63ce872640 1 -- 172.21.15.102:0/810768282 shutdown_connections 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.947+0000 7f63ce872640 1 --2- 172.21.15.102:0/810768282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63c8101030 0x7f63c810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.947+0000 7f63ce872640 1 --2- 172.21.15.102:0/810768282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63c8100670 0x7f63c8100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.947+0000 7f63ce872640 1 --2- 172.21.15.102:0/810768282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8106630 0x7f63c8106a10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.947+0000 7f63ce872640 1 -- 172.21.15.102:0/810768282 >> 172.21.15.102:0/810768282 conn(0x7f63c80777c0 msgr2=0x7f63c8077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.947+0000 7f63ce872640 1 -- 172.21.15.102:0/810768282 shutdown_connections 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.947+0000 7f63ce872640 1 -- 172.21.15.102:0/810768282 wait complete. 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63ce872640 1 Processor -- start 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63ce872640 1 -- start start 2024-04-24T23:19:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63ce872640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8100670 0x7f63c8196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63ce872640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63c8101030 0x7f63c8197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63ce872640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63c8106630 0x7f63c819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63ce872640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8110940 con 0x7f63c8100670 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63ce872640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81107c0 con 0x7f63c8106630 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63ce872640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8110ac0 con 0x7f63c8101030 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8100670 0x7f63c8196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8100670 0x7f63c8196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60246/0 (socket says 172.21.15.102:60246) 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c7fff640 1 -- 172.21.15.102:0/1458801959 learned_addr learned my addr 172.21.15.102:0/1458801959 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63cce0f640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63c8106630 0x7f63c819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c77fe640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63c8101030 0x7f63c8197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c7fff640 1 -- 172.21.15.102:0/1458801959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63c8101030 msgr2=0x7f63c8197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c7fff640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63c8101030 0x7f63c8197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c7fff640 1 -- 172.21.15.102:0/1458801959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63c8106630 msgr2=0x7f63c819b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c7fff640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63c8106630 0x7f63c819b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c7fff640 1 -- 172.21.15.102:0/1458801959 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c819be10 con 0x7f63c8100670 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.948+0000 7f63c77fe640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63c8101030 0x7f63c8197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.949+0000 7f63c7fff640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8100670 0x7f63c8196e20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005ab0 tx=0x7f63b8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.949+0000 7f63c57fa640 1 -- 172.21.15.102:0/1458801959 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b803f070 con 0x7f63c8100670 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.949+0000 7f63ce872640 1 -- 172.21.15.102:0/1458801959 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c819c0a0 con 0x7f63c8100670 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.949+0000 7f63c57fa640 1 -- 172.21.15.102:0/1458801959 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63b8031d60 con 0x7f63c8100670 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.949+0000 7f63c57fa640 1 -- 172.21.15.102:0/1458801959 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8035030 con 0x7f63c8100670 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.949+0000 7f63ce872640 1 -- 172.21.15.102:0/1458801959 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c8068df0 con 0x7f63c8100670 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.951+0000 7f63c57fa640 1 -- 172.21.15.102:0/1458801959 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f63b8042070 con 0x7f63c8100670 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.951+0000 7f63c57fa640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f63b00a2da0 0x7f63b00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63c57fa640 1 -- 172.21.15.102:0/1458801959 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f63b80efc50 con 0x7f63c8100670 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63c77fe640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f63b00a2da0 0x7f63b00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63d157c740 1 Processor -- start 2024-04-24T23:19:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63d157c740 1 -- start start 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63d157c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c71c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63d157c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c726cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63d157c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c729620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63d157c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cc6c680cb0 con 0x55cc6c727230 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63d157c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cc6c6295b0 con 0x55cc6c71cb60 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63d157c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cc6c67cc70 con 0x55cc6c71c240 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63c77fe640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f63b00a2da0 0x7f63b00a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0058c0 tx=0x7f63bc005850 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63cce0f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c729620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63cce0f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c729620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60250/0 (socket says 172.21.15.102:60250) 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63c7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c71c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63cce0f640 1 -- 172.21.15.102:0/2196995027 learned_addr learned my addr 172.21.15.102:0/2196995027 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.952+0000 7f63c77fe640 1 --2- 172.21.15.102:0/2196995027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c726cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63cce0f640 1 -- 172.21.15.102:0/2196995027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 msgr2=0x55cc6c71c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63cce0f640 1 --2- 172.21.15.102:0/2196995027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c71c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63cce0f640 1 -- 172.21.15.102:0/2196995027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 msgr2=0x55cc6c726cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63cce0f640 1 --2- 172.21.15.102:0/2196995027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c726cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63cce0f640 1 -- 172.21.15.102:0/2196995027 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6c395210 con 0x55cc6c727230 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63c77fe640 1 --2- 172.21.15.102:0/2196995027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c726cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63cce0f640 1 --2- 172.21.15.102:0/2196995027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c729620 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f63c0010300 tx=0x7f63c000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cebf2357927c00f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63ae7fc640 1 -- 172.21.15.102:0/2196995027 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c00094c0 con 0x55cc6c727230 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63ae7fc640 1 -- 172.21.15.102:0/2196995027 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63c001b070 con 0x55cc6c727230 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.953+0000 7f63ae7fc640 1 -- 172.21.15.102:0/2196995027 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0016a70 con 0x55cc6c727230 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 -- 172.21.15.102:0/2196995027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 msgr2=0x55cc6c729620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 --2- 172.21.15.102:0/2196995027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c729620 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f63c0010300 tx=0x7f63c000b860 comp rx=0 tx=0).stop 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 -- 172.21.15.102:0/2196995027 shutdown_connections 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 --2- 172.21.15.102:0/2196995027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c729620 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 --2- 172.21.15.102:0/2196995027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c726cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 --2- 172.21.15.102:0/2196995027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c71c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 -- 172.21.15.102:0/2196995027 >> 172.21.15.102:0/2196995027 conn(0x55cc6c718560 msgr2=0x55cc6c718940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 -- 172.21.15.102:0/2196995027 shutdown_connections 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 -- 172.21.15.102:0/2196995027 wait complete. 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 -- start start 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 Processor -- start 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c724120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.954+0000 7f63d157c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c724660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63d157c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c727230 0x55cc6c724ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63d157c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cc6c729fc0 con 0x55cc6c71cb60 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63d157c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cc6c63cb50 con 0x55cc6c727230 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63d157c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cc6c72a140 con 0x55cc6c71c240 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63d157c740 10 client.0 _ll_register_callbacks cb 0x55cc6c677ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c724660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c724660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60264/0 (socket says 172.21.15.102:60264) 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c77fe640 1 -- 172.21.15.102:0/1751040470 learned_addr learned my addr 172.21.15.102:0/1751040470 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63cce0f640 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c727230 0x55cc6c724ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c7fff640 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c724120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c77fe640 1 -- 172.21.15.102:0/1751040470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 msgr2=0x55cc6c724120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c77fe640 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c724120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c77fe640 1 -- 172.21.15.102:0/1751040470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c727230 msgr2=0x55cc6c724ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c77fe640 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c727230 0x55cc6c724ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c77fe640 1 -- 172.21.15.102:0/1751040470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc6c461620 con 0x55cc6c71cb60 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c7fff640 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c724120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63cce0f640 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c727230 0x55cc6c724ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63c77fe640 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c724660 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00ba80 tx=0x7f63bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c519d1842fb076d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63a77fe640 1 -- 172.21.15.102:0/1751040470 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc004220 con 0x55cc6c71cb60 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63a77fe640 1 -- 172.21.15.102:0/1751040470 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f63bc0043c0 con 0x55cc6c71cb60 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63d157c740 1 -- 172.21.15.102:0/1751040470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63c001b070 con 0x55cc6c71cb60 2024-04-24T23:19:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.955+0000 7f63a77fe640 1 -- 172.21.15.102:0/1751040470 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63bc01f700 con 0x55cc6c71cb60 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63d157c740 10 client.15948 fetch_fsmap learned FSMap version 26 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63d157c740 1 -- 172.21.15.102:0/1751040470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc6c736170 con 0x55cc6c71cb60 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63a77fe640 1 -- 172.21.15.102:0/1751040470 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63c001b070 con 0x55cc6c71cb60 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63a77fe640 1 -- 172.21.15.102:0/1751040470 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f63bc003bd0 con 0x55cc6c71cb60 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63d157c740 10 client.15948 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63d157c740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63d157c740 20 client.15948 populate_metadata read hostname 'smithi102' 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63d157c740 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55cc6c737680 0x55cc6c739a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63d157c740 4 client.15948 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63d157c740 1 -- 172.21.15.102:0/1751040470 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc6c68e1f0 con 0x55cc6c737680 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.956+0000 7f63c7fff640 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55cc6c737680 0x55cc6c739a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.957+0000 7f63c7fff640 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55cc6c737680 0x55cc6c739a90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69a6e8c2f395725 server_cookie=7e8a822574a4997 in_seq=0 out_seq=0 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.957+0000 7f63a77fe640 10 client.15948 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.957+0000 7f63a77fe640 1 -- 172.21.15.102:0/1751040470 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc6c68e1f0 con 0x55cc6c737680 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.957+0000 7f63a77fe640 10 client.15948 handle_command_reply: tid=0 2024-04-24T23:19:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.957+0000 7f63d157c740 1 client.15948 shutdown 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 -- 172.21.15.102:0/1751040470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 msgr2=0x55cc6c724660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c724660 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00ba80 tx=0x7f63bc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 -- 172.21.15.102:0/1751040470 shutdown_connections 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c727230 0x55cc6c724ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55cc6c737680 0x55cc6c739a90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c724660 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63a77fe640 0 client.15948 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 --2- 172.21.15.102:0/1751040470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c724120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 -- 172.21.15.102:0/1751040470 >> 172.21.15.102:0/1751040470 conn(0x55cc6c718560 msgr2=0x55cc6c71e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 -- 172.21.15.102:0/1751040470 shutdown_connections 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 1 -- 172.21.15.102:0/1751040470 wait complete. 2024-04-24T23:19:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.958+0000 7f63d157c740 20 client.15948 trim_cache size 0 max 16384 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63d157c740 1 Processor -- start 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63d157c740 1 -- start start 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63d157c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71c240 0x55cc6c732e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63d157c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c71eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63d157c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c727230 0x55cc6c7247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63d157c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81adae0 con 0x55cc6c71cb60 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63d157c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81ab3b0 con 0x55cc6c71c240 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63d157c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81a9220 con 0x55cc6c727230 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c71eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c71eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60274/0 (socket says 172.21.15.102:60274) 2024-04-24T23:19:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63c77fe640 1 -- 172.21.15.102:0/2527185789 learned_addr learned my addr 172.21.15.102:0/2527185789 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63c7fff640 1 --2- 172.21.15.102:0/2527185789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71c240 0x55cc6c732e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63c77fe640 1 -- 172.21.15.102:0/2527185789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c727230 msgr2=0x55cc6c7247f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63c77fe640 1 --2- 172.21.15.102:0/2527185789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c727230 0x55cc6c7247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63c77fe640 1 -- 172.21.15.102:0/2527185789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71c240 msgr2=0x55cc6c732e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63c77fe640 1 --2- 172.21.15.102:0/2527185789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71c240 0x55cc6c732e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.975+0000 7f63c77fe640 1 -- 172.21.15.102:0/2527185789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6c71c040 con 0x55cc6c71cb60 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.976+0000 7f63c7fff640 1 --2- 172.21.15.102:0/2527185789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71c240 0x55cc6c732e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.976+0000 7f63c77fe640 1 --2- 172.21.15.102:0/2527185789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c71eab0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f63bc000c00 tx=0x7f63bc00bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1308bab6f811f313 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.976+0000 7f63adffb640 1 -- 172.21.15.102:0/2527185789 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc03b020 con 0x55cc6c71cb60 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.976+0000 7f63adffb640 1 -- 172.21.15.102:0/2527185789 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63bc01f8a0 con 0x55cc6c71cb60 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.976+0000 7f63adffb640 1 -- 172.21.15.102:0/2527185789 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc033040 con 0x55cc6c71cb60 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 -- 172.21.15.102:0/2527185789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 msgr2=0x55cc6c71eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 --2- 172.21.15.102:0/2527185789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c71eab0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f63bc000c00 tx=0x7f63bc00bfa0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 -- 172.21.15.102:0/2527185789 shutdown_connections 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 --2- 172.21.15.102:0/2527185789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c727230 0x55cc6c7247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 --2- 172.21.15.102:0/2527185789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c71cb60 0x55cc6c71eab0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 --2- 172.21.15.102:0/2527185789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71c240 0x55cc6c732e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 -- 172.21.15.102:0/2527185789 >> 172.21.15.102:0/2527185789 conn(0x55cc6c718560 msgr2=0x55cc6c7240e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 -- 172.21.15.102:0/2527185789 shutdown_connections 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 -- 172.21.15.102:0/2527185789 wait complete. 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 -- start start 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 Processor -- start 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c72ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c72b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c72b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81a9870 con 0x55cc6c727230 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81aace0 con 0x55cc6c71cb60 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81a9f40 con 0x55cc6c71c240 2024-04-24T23:19:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.977+0000 7f63d157c740 10 client.0 _ll_register_callbacks cb 0x7f63bc015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63cce0f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c72b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63cce0f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c72b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60280/0 (socket says 172.21.15.102:60280) 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63cce0f640 1 -- 172.21.15.102:0/3669077591 learned_addr learned my addr 172.21.15.102:0/3669077591 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63c7fff640 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c72ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63c77fe640 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c72b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63cce0f640 1 -- 172.21.15.102:0/3669077591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 msgr2=0x55cc6c72ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63cce0f640 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c72ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63cce0f640 1 -- 172.21.15.102:0/3669077591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 msgr2=0x55cc6c72b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63cce0f640 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c72b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63cce0f640 1 -- 172.21.15.102:0/3669077591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc6c79cc90 con 0x55cc6c727230 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63c77fe640 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c72b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63cce0f640 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c72b8b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f63c00102a0 tx=0x7f63c0016b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9046056809042798 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:19:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63a77fe640 1 -- 172.21.15.102:0/3669077591 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0024070 con 0x55cc6c727230 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63a77fe640 1 -- 172.21.15.102:0/3669077591 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f63c00094c0 con 0x55cc6c727230 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63d157c740 1 -- 172.21.15.102:0/3669077591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63bc01f8a0 con 0x55cc6c727230 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63a77fe640 1 -- 172.21.15.102:0/3669077591 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63c00096c0 con 0x55cc6c727230 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63d157c740 10 client.15954 fetch_fsmap learned FSMap version 26 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.978+0000 7f63d157c740 1 -- 172.21.15.102:0/3669077591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc6c79def0 con 0x55cc6c727230 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.979+0000 7f63a77fe640 1 -- 172.21.15.102:0/3669077591 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63bc01f8a0 con 0x55cc6c727230 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.979+0000 7f63a77fe640 1 -- 172.21.15.102:0/3669077591 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f63c0029990 con 0x55cc6c727230 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.979+0000 7f63d157c740 10 client.15954 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.979+0000 7f63d157c740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.979+0000 7f63d157c740 20 client.15954 populate_metadata read hostname 'smithi102' 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.979+0000 7f63d157c740 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55cc6c79e410 0x55cc6c7a0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.979+0000 7f63d157c740 4 client.15954 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.979+0000 7f63d157c740 1 -- 172.21.15.102:0/3669077591 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc6c688540 con 0x55cc6c79e410 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.979+0000 7f63c7fff640 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55cc6c79e410 0x55cc6c7a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.980+0000 7f63a77fe640 10 client.15954 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.980+0000 7f63c7fff640 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55cc6c79e410 0x55cc6c7a0830 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d0b66f49b134a3 server_cookie=abd10aa1a2e0749d in_seq=0 out_seq=0 2024-04-24T23:19:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.987+0000 7f63a77fe640 1 -- 172.21.15.102:0/3669077591 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc6c688540 con 0x55cc6c79e410 2024-04-24T23:19:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.987+0000 7f63a77fe640 10 client.15954 handle_command_reply: tid=0 2024-04-24T23:19:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.987+0000 7f63d157c740 1 client.15954 shutdown 2024-04-24T23:19:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 -- 172.21.15.102:0/3669077591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 msgr2=0x55cc6c72b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c72b8b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f63c00102a0 tx=0x7f63c0016b90 comp rx=0 tx=0).stop 2024-04-24T23:19:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 -- 172.21.15.102:0/3669077591 shutdown_connections 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55cc6c79e410 0x55cc6c7a0830 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc6c727230 0x55cc6c72b8b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc6c71cb60 0x55cc6c72b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 --2- 172.21.15.102:0/3669077591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc6c71c240 0x55cc6c72ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63a77fe640 0 client.15954 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 -- 172.21.15.102:0/3669077591 >> 172.21.15.102:0/3669077591 conn(0x55cc6c718560 msgr2=0x55cc6c717440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 -- 172.21.15.102:0/3669077591 shutdown_connections 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 1 -- 172.21.15.102:0/3669077591 wait complete. 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.988+0000 7f63d157c740 20 client.15954 trim_cache size 0 max 16384 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:19:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:19:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:19:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:19:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 -- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f63b00a2da0 msgr2=0x7f63b00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f63b00a2da0 0x7f63b00a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0058c0 tx=0x7f63bc005850 comp rx=0 tx=0).stop 2024-04-24T23:19:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 -- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8100670 msgr2=0x7f63c8196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8100670 0x7f63c8196e20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005ab0 tx=0x7f63b8003cd0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 -- 172.21.15.102:0/1458801959 shutdown_connections 2024-04-24T23:19:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63c8106630 0x7f63c819b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63c8101030 0x7f63c8197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f63b00a2da0 0x7f63b00a5260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 --2- 172.21.15.102:0/1458801959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c8100670 0x7f63c8196e20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 -- 172.21.15.102:0/1458801959 >> 172.21.15.102:0/1458801959 conn(0x7f63c80777c0 msgr2=0x7f63c81031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 -- 172.21.15.102:0/1458801959 shutdown_connections 2024-04-24T23:19:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:19:51.989+0000 7f63ce872640 1 -- 172.21.15.102:0/1458801959 wait complete. 2024-04-24T23:19:53.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:53 smithi102 ceph-mon[29216]: pgmap v208: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:53.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:53 smithi161 ceph-mon[32709]: pgmap v208: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:53.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:53 smithi183 ceph-mon[33137]: pgmap v208: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:55.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:55 smithi102 ceph-mon[29216]: pgmap v209: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:19:55.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:55 smithi161 ceph-mon[32709]: pgmap v209: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:19:55.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:55 smithi183 ceph-mon[33137]: pgmap v209: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:19:57.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:57 smithi102 ceph-mon[29216]: pgmap v210: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:57 smithi161 ceph-mon[32709]: pgmap v210: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:57.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:57 smithi183 ceph-mon[33137]: pgmap v210: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:19:59.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:19:59 smithi183 ceph-mon[33137]: pgmap v211: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:59.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:19:59 smithi102 ceph-mon[29216]: pgmap v211: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:19:59.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:19:59 smithi161 ceph-mon[32709]: pgmap v211: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:00.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:00 smithi102 ceph-mon[29216]: overall HEALTH_OK 2024-04-24T23:20:00.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:00 smithi161 ceph-mon[32709]: overall HEALTH_OK 2024-04-24T23:20:00.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:00 smithi183 ceph-mon[33137]: overall HEALTH_OK 2024-04-24T23:20:01.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:01 smithi102 ceph-mon[29216]: pgmap v212: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:01.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:01 smithi161 ceph-mon[32709]: pgmap v212: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:01 smithi183 ceph-mon[33137]: pgmap v212: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:03.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:03 smithi102 ceph-mon[29216]: pgmap v213: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:03 smithi161 ceph-mon[32709]: pgmap v213: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:03 smithi183 ceph-mon[33137]: pgmap v213: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 -- 172.21.15.102:0/2631469097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0750103210 msgr2=0x7f075010b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 --2- 172.21.15.102:0/2631469097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0750103210 0x7f075010b6f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0740009900 tx=0x7f074002a480 comp rx=0 tx=0).stop 2024-04-24T23:20:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 -- 172.21.15.102:0/2631469097 shutdown_connections 2024-04-24T23:20:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 --2- 172.21.15.102:0/2631469097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f075010bd10 0x7f075010e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 --2- 172.21.15.102:0/2631469097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0750103210 0x7f075010b6f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 --2- 172.21.15.102:0/2631469097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07501028f0 0x7f0750102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 -- 172.21.15.102:0/2631469097 >> 172.21.15.102:0/2631469097 conn(0x7f07500fc110 msgr2=0x7f07500fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 -- 172.21.15.102:0/2631469097 shutdown_connections 2024-04-24T23:20:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 -- 172.21.15.102:0/2631469097 wait complete. 2024-04-24T23:20:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 Processor -- start 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.090+0000 7f0757bee640 1 -- start start 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0757bee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07501028f0 0x7f0750197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0757bee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0750103210 0x7f07501983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0757bee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f075010bd10 0x7f075019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0757bee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07501119c0 con 0x7f075010bd10 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0757bee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0750111840 con 0x7f07501028f0 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0757bee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0750111b40 con 0x7f0750103210 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075618b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f075010bd10 0x7f075019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075618b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f075010bd10 0x7f075019c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48530/0 (socket says 172.21.15.102:48530) 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075618b640 1 -- 172.21.15.102:0/1894182932 learned_addr learned my addr 172.21.15.102:0/1894182932 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075598a640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07501028f0 0x7f0750197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0755189640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0750103210 0x7f07501983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075618b640 1 -- 172.21.15.102:0/1894182932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0750103210 msgr2=0x7f07501983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075618b640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0750103210 0x7f07501983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075618b640 1 -- 172.21.15.102:0/1894182932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07501028f0 msgr2=0x7f0750197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075618b640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07501028f0 0x7f0750197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075618b640 1 -- 172.21.15.102:0/1894182932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075019ce90 con 0x7f075010bd10 2024-04-24T23:20:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f075618b640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f075010bd10 0x7f075019c760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f074800eb10 tx=0x7f074800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0746ffd640 1 -- 172.21.15.102:0/1894182932 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748004b20 con 0x7f075010bd10 2024-04-24T23:20:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0746ffd640 1 -- 172.21.15.102:0/1894182932 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0748004cc0 con 0x7f075010bd10 2024-04-24T23:20:05.097 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0746ffd640 1 -- 172.21.15.102:0/1894182932 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748010690 con 0x7f075010bd10 2024-04-24T23:20:05.097 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0757bee640 1 -- 172.21.15.102:0/1894182932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075019d180 con 0x7f075010bd10 2024-04-24T23:20:05.097 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.091+0000 7f0757bee640 1 -- 172.21.15.102:0/1894182932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501a4a10 con 0x7f075010bd10 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.094+0000 7f0746ffd640 1 -- 172.21.15.102:0/1894182932 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0748005000 con 0x7f075010bd10 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075a9d8740 1 Processor -- start 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075a9d8740 1 -- start start 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988f120 0x5620f988f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988fa40 0x5620f989e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f0746ffd640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f073c0a2e20 0x7f073c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f98a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075a9d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5620f97f3cb0 con 0x5620f989e640 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075a9d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620f979c5b0 con 0x5620f988f120 2024-04-24T23:20:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075a9d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5620f97efc70 con 0x5620f988fa40 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075618b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f98a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075598a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988fa40 0x5620f989e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075618b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f98a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48540/0 (socket says 172.21.15.102:48540) 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f0755189640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f073c0a2e20 0x7f073c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f075618b640 1 -- 172.21.15.102:0/4113756627 learned_addr learned my addr 172.21.15.102:0/4113756627 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f0755189640 1 --2- 172.21.15.102:0/4113756627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988f120 0x5620f988f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.095+0000 7f0746ffd640 1 -- 172.21.15.102:0/1894182932 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f07480cf970 con 0x7f075010bd10 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f075598a640 1 -- 172.21.15.102:0/4113756627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988f120 msgr2=0x5620f988f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f075598a640 1 --2- 172.21.15.102:0/4113756627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988f120 0x5620f988f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f075598a640 1 -- 172.21.15.102:0/4113756627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 msgr2=0x5620f98a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f075598a640 1 --2- 172.21.15.102:0/4113756627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f98a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f075598a640 1 -- 172.21.15.102:0/4113756627 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f9508210 con 0x5620f988fa40 2024-04-24T23:20:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f075618b640 1 --2- 172.21.15.102:0/4113756627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f98a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f0755189640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f073c0a2e20 0x7f073c0a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f074002a9a0 tx=0x7f0740031400 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f075598a640 1 --2- 172.21.15.102:0/4113756627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988fa40 0x5620f989e100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f074c009c20 tx=0x7f074c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9981c37dc9599b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f073bfff640 1 -- 172.21.15.102:0/4113756627 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c004050 con 0x5620f988fa40 2024-04-24T23:20:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f073bfff640 1 -- 172.21.15.102:0/4113756627 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f074c0054d0 con 0x5620f988fa40 2024-04-24T23:20:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.096+0000 7f073bfff640 1 -- 172.21.15.102:0/4113756627 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c004cf0 con 0x5620f988fa40 2024-04-24T23:20:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 -- 172.21.15.102:0/4113756627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988fa40 msgr2=0x5620f989e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 --2- 172.21.15.102:0/4113756627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988fa40 0x5620f989e100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f074c009c20 tx=0x7f074c011810 comp rx=0 tx=0).stop 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 -- 172.21.15.102:0/4113756627 shutdown_connections 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 --2- 172.21.15.102:0/4113756627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f98a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 --2- 172.21.15.102:0/4113756627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988fa40 0x5620f989e100 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 --2- 172.21.15.102:0/4113756627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988f120 0x5620f988f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 -- 172.21.15.102:0/4113756627 >> 172.21.15.102:0/4113756627 conn(0x5620f988b560 msgr2=0x5620f988b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 -- 172.21.15.102:0/4113756627 shutdown_connections 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 -- 172.21.15.102:0/4113756627 wait complete. 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 -- start start 2024-04-24T23:20:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.097+0000 7f075a9d8740 1 Processor -- start 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 0x5620f989b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 0x5620f989b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f989bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075a9d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5620f98a1480 con 0x5620f989e640 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075a9d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620f97afb50 con 0x5620f988fa40 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075a9d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5620f98a1600 con 0x5620f988f120 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075a9d8740 10 client.0 _ll_register_callbacks cb 0x5620f97eaca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075618b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f989bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075598a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 0x5620f989b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075618b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f989bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48550/0 (socket says 172.21.15.102:48550) 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075618b640 1 -- 172.21.15.102:0/2715771352 learned_addr learned my addr 172.21.15.102:0/2715771352 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f0755189640 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 0x5620f989b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075618b640 1 -- 172.21.15.102:0/2715771352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 msgr2=0x5620f989b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075618b640 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 0x5620f989b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075618b640 1 -- 172.21.15.102:0/2715771352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 msgr2=0x5620f989b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075618b640 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 0x5620f989b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075618b640 1 -- 172.21.15.102:0/2715771352 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f95d4620 con 0x5620f989e640 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f075618b640 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f989bf30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f074809bb80 tx=0x7f074809ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d42840cdece289b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f07397fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f07397fa640 1 -- 172.21.15.102:0/2715771352 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074809be80 con 0x5620f989e640 2024-04-24T23:20:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.098+0000 7f07397fa640 1 -- 172.21.15.102:0/2715771352 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f07480765f0 con 0x5620f989e640 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075a9d8740 1 -- 172.21.15.102:0/2715771352 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c0054d0 con 0x5620f989e640 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f07397fa640 1 -- 172.21.15.102:0/2715771352 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07480a0230 con 0x5620f989e640 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075a9d8740 10 client.15966 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075a9d8740 1 -- 172.21.15.102:0/2715771352 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5620f98ad670 con 0x5620f989e640 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f07397fa640 1 -- 172.21.15.102:0/2715771352 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f074c0054d0 con 0x5620f989e640 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f07397fa640 1 -- 172.21.15.102:0/2715771352 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f07480a4660 con 0x5620f989e640 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075a9d8740 10 client.15966 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075a9d8740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075a9d8740 20 client.15966 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075a9d8740 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5620f98aeb30 0x5620f98b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075a9d8740 4 client.15966 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075a9d8740 1 -- 172.21.15.102:0/2715771352 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620f98011f0 con 0x5620f98aeb30 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.099+0000 7f075598a640 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5620f98aeb30 0x5620f98b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.100+0000 7f07397fa640 10 client.15966 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.100+0000 7f075598a640 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5620f98aeb30 0x5620f98b0f20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65014fd45e3c3f21 server_cookie=c57b0d218d4d4b3d in_seq=0 out_seq=0 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f07397fa640 1 -- 172.21.15.102:0/2715771352 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620f98011f0 con 0x5620f98aeb30 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f07397fa640 10 client.15966 handle_command_reply: tid=0 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 client.15966 shutdown 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 -- 172.21.15.102:0/2715771352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 msgr2=0x5620f989bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f989bf30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f074809bb80 tx=0x7f074809ba80 comp rx=0 tx=0).stop 2024-04-24T23:20:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 -- 172.21.15.102:0/2715771352 shutdown_connections 2024-04-24T23:20:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5620f98aeb30 0x5620f98b0f20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f989e640 0x5620f989bf30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 0x5620f989b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f07397fa640 0 client.15966 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 --2- 172.21.15.102:0/2715771352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 0x5620f989b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 -- 172.21.15.102:0/2715771352 >> 172.21.15.102:0/2715771352 conn(0x5620f988b560 msgr2=0x5620f9891310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 -- 172.21.15.102:0/2715771352 shutdown_connections 2024-04-24T23:20:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 1 -- 172.21.15.102:0/2715771352 wait complete. 2024-04-24T23:20:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.101+0000 7f075a9d8740 20 client.15966 trim_cache size 0 max 16384 2024-04-24T23:20:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.118+0000 7f075a9d8740 1 Processor -- start 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075a9d8740 1 -- start start 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988f120 0x5620f989dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 0x5620f98aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f989e640 0x5620f988a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075a9d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ada70 con 0x5620f988f120 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075a9d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ab340 con 0x5620f988fa40 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075a9d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07501a91b0 con 0x5620f989e640 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075598a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988f120 0x5620f989dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075598a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988f120 0x5620f989dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48558/0 (socket says 172.21.15.102:48558) 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075598a640 1 -- 172.21.15.102:0/840006308 learned_addr learned my addr 172.21.15.102:0/840006308 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075618b640 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f989e640 0x5620f988a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f0755189640 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 0x5620f98aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075598a640 1 -- 172.21.15.102:0/840006308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f989e640 msgr2=0x5620f988a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075598a640 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f989e640 0x5620f988a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075598a640 1 -- 172.21.15.102:0/840006308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 msgr2=0x5620f98aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075598a640 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 0x5620f98aa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075598a640 1 -- 172.21.15.102:0/840006308 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f98962d0 con 0x5620f988f120 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075618b640 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f989e640 0x5620f988a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f0755189640 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 0x5620f98aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.119+0000 7f075598a640 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988f120 0x5620f989dbe0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f074c014040 tx=0x7f074c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f227bab7540baa12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f073b7fe640 1 -- 172.21.15.102:0/840006308 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c02edd0 con 0x5620f988f120 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f073b7fe640 1 -- 172.21.15.102:0/840006308 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f074c02e860 con 0x5620f988f120 2024-04-24T23:20:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f073b7fe640 1 -- 172.21.15.102:0/840006308 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c003850 con 0x5620f988f120 2024-04-24T23:20:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f075a9d8740 1 -- 172.21.15.102:0/840006308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988f120 msgr2=0x5620f989dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f075a9d8740 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988f120 0x5620f989dbe0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f074c014040 tx=0x7f074c009680 comp rx=0 tx=0).stop 2024-04-24T23:20:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f075a9d8740 1 -- 172.21.15.102:0/840006308 shutdown_connections 2024-04-24T23:20:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f075a9d8740 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f989e640 0x5620f988a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f075a9d8740 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f988fa40 0x5620f98aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f075a9d8740 1 --2- 172.21.15.102:0/840006308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988f120 0x5620f989dbe0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.120+0000 7f075a9d8740 1 -- 172.21.15.102:0/840006308 >> 172.21.15.102:0/840006308 conn(0x5620f988b560 msgr2=0x5620f9889d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 -- 172.21.15.102:0/840006308 shutdown_connections 2024-04-24T23:20:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 -- 172.21.15.102:0/840006308 wait complete. 2024-04-24T23:20:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 -- start start 2024-04-24T23:20:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 Processor -- start 2024-04-24T23:20:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 0x5620f98a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988fa40 0x5620f9897e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f989e640 0x5620f98983a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07501a9800 con 0x5620f988fa40 2024-04-24T23:20:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f07501aac70 con 0x5620f989e640 2024-04-24T23:20:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07501a9ed0 con 0x5620f988f120 2024-04-24T23:20:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f075a9d8740 10 client.0 _ll_register_callbacks cb 0x5620f989e590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.121+0000 7f0755189640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988fa40 0x5620f9897e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f0755189640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988fa40 0x5620f9897e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48566/0 (socket says 172.21.15.102:48566) 2024-04-24T23:20:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f0755189640 1 -- 172.21.15.102:0/3726709525 learned_addr learned my addr 172.21.15.102:0/3726709525 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f075598a640 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 0x5620f98a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f075618b640 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f989e640 0x5620f98983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f0755189640 1 -- 172.21.15.102:0/3726709525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 msgr2=0x5620f98a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f0755189640 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 0x5620f98a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f0755189640 1 -- 172.21.15.102:0/3726709525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f989e640 msgr2=0x5620f98983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f0755189640 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f989e640 0x5620f98983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f0755189640 1 -- 172.21.15.102:0/3726709525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f9898bd0 con 0x5620f988fa40 2024-04-24T23:20:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f075618b640 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f989e640 0x5620f98983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f0755189640 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988fa40 0x5620f9897e60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0740009900 tx=0x7f074002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a225b8022eca61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f07397fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f07397fa640 1 -- 172.21.15.102:0/3726709525 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074004f070 con 0x5620f988fa40 2024-04-24T23:20:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f075a9d8740 1 -- 172.21.15.102:0/3726709525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c02e860 con 0x5620f988fa40 2024-04-24T23:20:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f07397fa640 1 -- 172.21.15.102:0/3726709525 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8535+0+0 (secure 0 0 0) 0x7f0740005460 con 0x5620f988fa40 2024-04-24T23:20:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f07397fa640 1 -- 172.21.15.102:0/3726709525 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0740035040 con 0x5620f988fa40 2024-04-24T23:20:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f075a9d8740 10 client.15972 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.122+0000 7f075a9d8740 1 -- 172.21.15.102:0/3726709525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5620f9915160 con 0x5620f988fa40 2024-04-24T23:20:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.123+0000 7f07397fa640 1 -- 172.21.15.102:0/3726709525 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f074c02e860 con 0x5620f988fa40 2024-04-24T23:20:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.123+0000 7f07397fa640 1 -- 172.21.15.102:0/3726709525 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f074003a040 con 0x5620f988fa40 2024-04-24T23:20:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.123+0000 7f075a9d8740 10 client.15972 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.123+0000 7f075a9d8740 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.123+0000 7f075a9d8740 20 client.15972 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.123+0000 7f075a9d8740 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5620f99158f0 0x5620f9917d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.123+0000 7f075a9d8740 4 client.15972 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:20:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.123+0000 7f075a9d8740 1 -- 172.21.15.102:0/3726709525 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620f97fb540 con 0x5620f99158f0 2024-04-24T23:20:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.123+0000 7f075598a640 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5620f99158f0 0x5620f9917d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.124+0000 7f07397fa640 10 client.15972 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.124+0000 7f075598a640 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5620f99158f0 0x5620f9917d10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45827e425e5592f9 server_cookie=3de12e179dae5db6 in_seq=0 out_seq=0 2024-04-24T23:20:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f07397fa640 1 -- 172.21.15.102:0/3726709525 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620f97fb540 con 0x5620f99158f0 2024-04-24T23:20:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f07397fa640 10 client.15972 handle_command_reply: tid=0 2024-04-24T23:20:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f075a9d8740 1 client.15972 shutdown 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f075a9d8740 1 -- 172.21.15.102:0/3726709525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988fa40 msgr2=0x5620f9897e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f075a9d8740 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988fa40 0x5620f9897e60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0740009900 tx=0x7f074002a480 comp rx=0 tx=0).stop 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f075a9d8740 1 -- 172.21.15.102:0/3726709525 shutdown_connections 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f075a9d8740 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620f989e640 0x5620f98983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f075a9d8740 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5620f99158f0 0x5620f9917d10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f075a9d8740 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620f988fa40 0x5620f9897e60 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f07397fa640 0 client.15972 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.132+0000 7f075a9d8740 1 --2- 172.21.15.102:0/3726709525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f988f120 0x5620f98a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f075a9d8740 1 -- 172.21.15.102:0/3726709525 >> 172.21.15.102:0/3726709525 conn(0x5620f988b560 msgr2=0x5620f988a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f075a9d8740 1 -- 172.21.15.102:0/3726709525 shutdown_connections 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f075a9d8740 1 -- 172.21.15.102:0/3726709525 wait complete. 2024-04-24T23:20:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f075a9d8740 20 client.15972 trim_cache size 0 max 16384 2024-04-24T23:20:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:20:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:20:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:20:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:20:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 -- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f073c0a2e20 msgr2=0x7f073c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f073c0a2e20 0x7f073c0a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f074002a9a0 tx=0x7f0740031400 comp rx=0 tx=0).stop 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 -- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f075010bd10 msgr2=0x7f075019c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f075010bd10 0x7f075019c760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f074800eb10 tx=0x7f074800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 -- 172.21.15.102:0/1894182932 shutdown_connections 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f073c0a2e20 0x7f073c0a52e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f075010bd10 0x7f075019c760 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0750103210 0x7f07501983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 --2- 172.21.15.102:0/1894182932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07501028f0 0x7f0750197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 -- 172.21.15.102:0/1894182932 >> 172.21.15.102:0/1894182932 conn(0x7f07500fc110 msgr2=0x7f07500fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 -- 172.21.15.102:0/1894182932 shutdown_connections 2024-04-24T23:20:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:05.133+0000 7f0757bee640 1 -- 172.21.15.102:0/1894182932 wait complete. 2024-04-24T23:20:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:05 smithi161 ceph-mon[32709]: pgmap v214: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 MiB/s wr, 42 op/s 2024-04-24T23:20:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:05 smithi183 ceph-mon[33137]: pgmap v214: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 MiB/s wr, 42 op/s 2024-04-24T23:20:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:05 smithi102 ceph-mon[29216]: pgmap v214: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 MiB/s wr, 42 op/s 2024-04-24T23:20:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:07 smithi161 ceph-mon[32709]: pgmap v215: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:20:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:07 smithi161 ceph-mon[32709]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:20:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:07 smithi183 ceph-mon[33137]: pgmap v215: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:20:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:07 smithi183 ceph-mon[33137]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:20:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:07 smithi102 ceph-mon[29216]: pgmap v215: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 MiB/s wr, 32 op/s 2024-04-24T23:20:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:07 smithi102 ceph-mon[29216]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:20:09.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:09 smithi161 ceph-mon[32709]: pgmap v217: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 41 op/s 2024-04-24T23:20:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:09 smithi183 ceph-mon[33137]: pgmap v217: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 41 op/s 2024-04-24T23:20:09.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:09 smithi102 ceph-mon[29216]: pgmap v217: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 41 op/s 2024-04-24T23:20:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:10 smithi183 ceph-mon[33137]: pgmap v218: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 41 op/s 2024-04-24T23:20:11.263 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:10 smithi183 ceph-mon[33137]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:20:11.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:10 smithi102 ceph-mon[29216]: pgmap v218: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 41 op/s 2024-04-24T23:20:11.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:10 smithi102 ceph-mon[29216]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:20:11.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:10 smithi161 ceph-mon[32709]: pgmap v218: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 41 op/s 2024-04-24T23:20:11.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:10 smithi161 ceph-mon[32709]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:20:13.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:12 smithi183 ceph-mon[33137]: pgmap v220: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 14 op/s 2024-04-24T23:20:13.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:12 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:12 smithi102 ceph-mon[29216]: pgmap v220: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 14 op/s 2024-04-24T23:20:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:12 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:13.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:12 smithi161 ceph-mon[32709]: pgmap v220: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 14 op/s 2024-04-24T23:20:13.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:12 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:14.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:13 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:14.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:13 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:14.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:13 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:20:14.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:13 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:14.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:13 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:14.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:13 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:20:14.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:13 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:14.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:13 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:14.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:13 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:20:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:15 smithi183 ceph-mon[33137]: pgmap v221: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:20:15.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:15 smithi102 ceph-mon[29216]: pgmap v221: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:20:15.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:15 smithi161 ceph-mon[32709]: pgmap v221: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:20:17.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:17 smithi183 ceph-mon[33137]: pgmap v222: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-04-24T23:20:17.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:17 smithi102 ceph-mon[29216]: pgmap v222: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-04-24T23:20:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:17 smithi161 ceph-mon[32709]: pgmap v222: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-04-24T23:20:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.230+0000 7f57c67c4640 1 -- 172.21.15.102:0/2938887955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0100670 msgr2=0x7f57c0100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.230+0000 7f57c67c4640 1 --2- 172.21.15.102:0/2938887955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0100670 0x7f57c0100af0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009930 tx=0x7f57b402a420 comp rx=0 tx=0).stop 2024-04-24T23:20:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.230+0000 7f57c67c4640 1 -- 172.21.15.102:0/2938887955 shutdown_connections 2024-04-24T23:20:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.230+0000 7f57c67c4640 1 --2- 172.21.15.102:0/2938887955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c0101030 0x7f57c010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.230+0000 7f57c67c4640 1 --2- 172.21.15.102:0/2938887955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0100670 0x7f57c0100af0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.230+0000 7f57c67c4640 1 --2- 172.21.15.102:0/2938887955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57c0106630 0x7f57c0106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.230+0000 7f57c67c4640 1 -- 172.21.15.102:0/2938887955 >> 172.21.15.102:0/2938887955 conn(0x7f57c00777c0 msgr2=0x7f57c0077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.230+0000 7f57c67c4640 1 -- 172.21.15.102:0/2938887955 shutdown_connections 2024-04-24T23:20:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.230+0000 7f57c67c4640 1 -- 172.21.15.102:0/2938887955 wait complete. 2024-04-24T23:20:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c67c4640 1 Processor -- start 2024-04-24T23:20:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c67c4640 1 -- start start 2024-04-24T23:20:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c67c4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c0100670 0x7f57c0196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c67c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57c0101030 0x7f57c0197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c67c4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0106630 0x7f57c0197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c67c4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0110940 con 0x7f57c0106630 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c67c4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01107c0 con 0x7f57c0100670 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c67c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0110ac0 con 0x7f57c0101030 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c4d61640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0106630 0x7f57c0197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c4d61640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0106630 0x7f57c0197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38034/0 (socket says 172.21.15.102:38034) 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57c4d61640 1 -- 172.21.15.102:0/1335019009 learned_addr learned my addr 172.21.15.102:0/1335019009 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.231+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57c0101030 0x7f57c0197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57bffff640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c0100670 0x7f57c0196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57c4d61640 1 -- 172.21.15.102:0/1335019009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57c0101030 msgr2=0x7f57c0197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57c4d61640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57c0101030 0x7f57c0197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57c4d61640 1 -- 172.21.15.102:0/1335019009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c0100670 msgr2=0x7f57c0196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57c4d61640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c0100670 0x7f57c0196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57c4d61640 1 -- 172.21.15.102:0/1335019009 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c01a6380 con 0x7f57c0106630 2024-04-24T23:20:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57c4d61640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0106630 0x7f57c0197590 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f57b800eaa0 tx=0x7f57b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57bd7fa640 1 -- 172.21.15.102:0/1335019009 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b800cdb0 con 0x7f57c0106630 2024-04-24T23:20:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57bd7fa640 1 -- 172.21.15.102:0/1335019009 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b8005020 con 0x7f57c0106630 2024-04-24T23:20:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57c67c4640 1 -- 172.21.15.102:0/1335019009 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c01a6670 con 0x7f57c0106630 2024-04-24T23:20:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57bd7fa640 1 -- 172.21.15.102:0/1335019009 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b80116d0 con 0x7f57c0106630 2024-04-24T23:20:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.232+0000 7f57c67c4640 1 -- 172.21.15.102:0/1335019009 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c01a6b40 con 0x7f57c0106630 2024-04-24T23:20:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57bd7fa640 1 -- 172.21.15.102:0/1335019009 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f57b80051d0 con 0x7f57c0106630 2024-04-24T23:20:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57c95a8740 1 Processor -- start 2024-04-24T23:20:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57c95a8740 1 -- start start 2024-04-24T23:20:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57c95a8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2d160bcb0 con 0x55d2d16a7b60 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57c95a8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d2d15b45b0 con 0x55d2d16b2230 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57c95a8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2d1607c70 con 0x55d2d16a7240 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57bffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57bd7fa640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f57a80a2da0 0x7f57a80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57bffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38036/0 (socket says 172.21.15.102:38036) 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57bffff640 1 -- 172.21.15.102:0/729105906 learned_addr learned my addr 172.21.15.102:0/729105906 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/729105906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/729105906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.235+0000 7f57c4d61640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f57a80a2da0 0x7f57a80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57bd7fa640 1 -- 172.21.15.102:0/1335019009 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7f57b80cf5f0 con 0x7f57c0106630 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57bffff640 1 -- 172.21.15.102:0/729105906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 msgr2=0x55d2d16a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57bffff640 1 --2- 172.21.15.102:0/729105906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16a7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57bffff640 1 -- 172.21.15.102:0/729105906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 msgr2=0x55d2d16b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57bffff640 1 --2- 172.21.15.102:0/729105906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57bffff640 1 -- 172.21.15.102:0/729105906 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d1320210 con 0x55d2d16a7b60 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57bffff640 1 --2- 172.21.15.102:0/729105906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16b1cf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f57b000b710 tx=0x7f57b0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b542e24f1f269ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57c4d61640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f57a80a2da0 0x7f57a80a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f57b80ccdb0 tx=0x7f57b806aab0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57a67fc640 1 -- 172.21.15.102:0/729105906 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0003b50 con 0x55d2d16a7b60 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57a67fc640 1 -- 172.21.15.102:0/729105906 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b0015070 con 0x55d2d16a7b60 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.236+0000 7f57a67fc640 1 -- 172.21.15.102:0/729105906 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0004cb0 con 0x55d2d16a7b60 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 -- 172.21.15.102:0/729105906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 msgr2=0x55d2d16b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 --2- 172.21.15.102:0/729105906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16b1cf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f57b000b710 tx=0x7f57b0004610 comp rx=0 tx=0).stop 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 -- 172.21.15.102:0/729105906 shutdown_connections 2024-04-24T23:20:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 --2- 172.21.15.102:0/729105906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 --2- 172.21.15.102:0/729105906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16b1cf0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 --2- 172.21.15.102:0/729105906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 -- 172.21.15.102:0/729105906 >> 172.21.15.102:0/729105906 conn(0x55d2d16a3560 msgr2=0x55d2d16a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 -- 172.21.15.102:0/729105906 shutdown_connections 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 -- 172.21.15.102:0/729105906 wait complete. 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.237+0000 7f57c95a8740 1 -- start start 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57c95a8740 1 Processor -- start 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57c95a8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2d16b5020 con 0x55d2d16a7b60 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57c95a8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d2d15c7b50 con 0x55d2d16b2230 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57c95a8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2d16b51a0 con 0x55d2d16a7240 2024-04-24T23:20:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57c95a8740 10 client.0 _ll_register_callbacks cb 0x55d2d1602ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38044/0 (socket says 172.21.15.102:38044) 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bffff640 1 -- 172.21.15.102:0/2565303268 learned_addr learned my addr 172.21.15.102:0/2565303268 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bffff640 1 -- 172.21.15.102:0/2565303268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 msgr2=0x55d2d16af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bffff640 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bffff640 1 -- 172.21.15.102:0/2565303268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 msgr2=0x55d2d16afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bffff640 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bffff640 1 -- 172.21.15.102:0/2565303268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d13ec620 con 0x55d2d16a7b60 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f57bffff640 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16af6b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f57b00039c0 tx=0x7f57b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9dd54975c6ade3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.238+0000 7f579f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f579f7fe640 1 -- 172.21.15.102:0/2565303268 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0028060 con 0x55d2d16a7b60 2024-04-24T23:20:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f579f7fe640 1 -- 172.21.15.102:0/2565303268 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7f57b0011440 con 0x55d2d16a7b60 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f57c95a8740 1 -- 172.21.15.102:0/2565303268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57b0015070 con 0x55d2d16a7b60 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f579f7fe640 1 -- 172.21.15.102:0/2565303268 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57b0011640 con 0x55d2d16a7b60 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f57c95a8740 10 client.15984 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f57c95a8740 1 -- 172.21.15.102:0/2565303268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2d16c11e0 con 0x55d2d16a7b60 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f579f7fe640 1 -- 172.21.15.102:0/2565303268 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b0015070 con 0x55d2d16a7b60 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f579f7fe640 1 -- 172.21.15.102:0/2565303268 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f57b0022070 con 0x55d2d16a7b60 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f57c95a8740 10 client.15984 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f57c95a8740 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f57c95a8740 20 client.15984 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f57c95a8740 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55d2d16c2590 0x55d2d16c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f57c95a8740 4 client.15984 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.239+0000 7f57c95a8740 1 -- 172.21.15.102:0/2565303268 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2d16191f0 con 0x55d2d16c2590 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.240+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55d2d16c2590 0x55d2d16c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.240+0000 7f579f7fe640 10 client.15984 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.240+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55d2d16c2590 0x55d2d16c4980 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75380de69152b89c server_cookie=c1798d3d206c6194 in_seq=0 out_seq=0 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f579f7fe640 1 -- 172.21.15.102:0/2565303268 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2d16191f0 con 0x55d2d16c2590 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f579f7fe640 10 client.15984 handle_command_reply: tid=0 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 client.15984 shutdown 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 -- 172.21.15.102:0/2565303268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 msgr2=0x55d2d16af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16af6b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f57b00039c0 tx=0x7f57b0004750 comp rx=0 tx=0).stop 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 -- 172.21.15.102:0/2565303268 shutdown_connections 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55d2d16c2590 0x55d2d16c4980 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16af6b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 --2- 172.21.15.102:0/2565303268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f579f7fe640 0 client.15984 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 -- 172.21.15.102:0/2565303268 >> 172.21.15.102:0/2565303268 conn(0x55d2d16a3560 msgr2=0x55d2d16a9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 -- 172.21.15.102:0/2565303268 shutdown_connections 2024-04-24T23:20:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 1 -- 172.21.15.102:0/2565303268 wait complete. 2024-04-24T23:20:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.241+0000 7f57c95a8740 20 client.15984 trim_cache size 0 max 16384 2024-04-24T23:20:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57c95a8740 1 Processor -- start 2024-04-24T23:20:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57c95a8740 1 -- start start 2024-04-24T23:20:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57c95a8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01b1f20 con 0x55d2d16a7b60 2024-04-24T23:20:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57c95a8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01af7f0 con 0x55d2d16b2230 2024-04-24T23:20:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57c95a8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01ad660 con 0x55d2d16a7240 2024-04-24T23:20:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57bffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57bffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38054/0 (socket says 172.21.15.102:38054) 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57bffff640 1 -- 172.21.15.102:0/168138747 learned_addr learned my addr 172.21.15.102:0/168138747 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.263+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57bffff640 1 -- 172.21.15.102:0/168138747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 msgr2=0x55d2d16bdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57bffff640 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16bdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57bffff640 1 -- 172.21.15.102:0/168138747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 msgr2=0x55d2d16ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57bffff640 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57bffff640 1 -- 172.21.15.102:0/168138747 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d16a7040 con 0x55d2d16a7b60 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57bffff640 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16a9ab0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f57b002d040 tx=0x7f57b00115d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09b0477001b8574 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57a5ffb640 1 -- 172.21.15.102:0/168138747 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0028070 con 0x55d2d16a7b60 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57a5ffb640 1 -- 172.21.15.102:0/168138747 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b0015070 con 0x55d2d16a7b60 2024-04-24T23:20:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.264+0000 7f57a5ffb640 1 -- 172.21.15.102:0/168138747 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0011ba0 con 0x55d2d16a7b60 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 -- 172.21.15.102:0/168138747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 msgr2=0x55d2d16a9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16a9ab0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f57b002d040 tx=0x7f57b00115d0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 -- 172.21.15.102:0/168138747 shutdown_connections 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16b2230 0x55d2d16ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16a7b60 0x55d2d16a9ab0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 --2- 172.21.15.102:0/168138747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7240 0x55d2d16bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 -- 172.21.15.102:0/168138747 >> 172.21.15.102:0/168138747 conn(0x55d2d16a3560 msgr2=0x55d2d16af130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 -- 172.21.15.102:0/168138747 shutdown_connections 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 -- 172.21.15.102:0/168138747 wait complete. 2024-04-24T23:20:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 -- start start 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.265+0000 7f57c95a8740 1 Processor -- start 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16a7240 0x55d2d16ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7b60 0x55d2d16ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57c95a8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16b2230 0x55d2d16abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57c95a8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01adcb0 con 0x55d2d16b2230 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57c95a8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01af120 con 0x55d2d16a7240 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57c95a8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01ae380 con 0x55d2d16a7b60 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57c95a8740 10 client.0 _ll_register_callbacks cb 0x7f57b40095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16b2230 0x55d2d16abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16b2230 0x55d2d16abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38066/0 (socket says 172.21.15.102:38066) 2024-04-24T23:20:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 -- 172.21.15.102:0/1265283478 learned_addr learned my addr 172.21.15.102:0/1265283478 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16a7240 0x55d2d16ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bffff640 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7b60 0x55d2d16ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 -- 172.21.15.102:0/1265283478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7b60 msgr2=0x55d2d16ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7b60 0x55d2d16ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 -- 172.21.15.102:0/1265283478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16a7240 msgr2=0x55d2d16ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16a7240 0x55d2d16ba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 -- 172.21.15.102:0/1265283478 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d16ac460 con 0x55d2d16b2230 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f57bf7fe640 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16b2230 0x55d2d16abf20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f57b4005ab0 tx=0x7f57b403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c3d2089d181c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f579f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.266+0000 7f579f7fe640 1 -- 172.21.15.102:0/1265283478 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4053060 con 0x55d2d16b2230 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f579f7fe640 1 -- 172.21.15.102:0/1265283478 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7f57b4003a30 con 0x55d2d16b2230 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57c95a8740 1 -- 172.21.15.102:0/1265283478 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57b0015070 con 0x55d2d16b2230 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f579f7fe640 1 -- 172.21.15.102:0/1265283478 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57b4035040 con 0x55d2d16b2230 2024-04-24T23:20:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57c95a8740 10 client.15990 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57c95a8740 1 -- 172.21.15.102:0/1265283478 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2d17284f0 con 0x55d2d16b2230 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f579f7fe640 1 -- 172.21.15.102:0/1265283478 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b0015070 con 0x55d2d16b2230 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f579f7fe640 1 -- 172.21.15.102:0/1265283478 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f57b404dce0 con 0x55d2d16b2230 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57c95a8740 10 client.15990 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57c95a8740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57c95a8740 20 client.15990 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57c95a8740 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55d2d1728bf0 0x55d2d172b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57c95a8740 4 client.15990 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57c95a8740 1 -- 172.21.15.102:0/1265283478 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2d1613540 con 0x55d2d1728bf0 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.267+0000 7f57bffff640 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55d2d1728bf0 0x55d2d172b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.268+0000 7f57bffff640 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55d2d1728bf0 0x55d2d172b010 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3096c23030f36703 server_cookie=9c69a27344ac7bdf in_seq=0 out_seq=0 2024-04-24T23:20:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.268+0000 7f579f7fe640 10 client.15990 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f579f7fe640 1 -- 172.21.15.102:0/1265283478 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2d1613540 con 0x55d2d1728bf0 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f579f7fe640 10 client.15990 handle_command_reply: tid=0 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 client.15990 shutdown 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 -- 172.21.15.102:0/1265283478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16b2230 msgr2=0x55d2d16abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16b2230 0x55d2d16abf20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f57b4005ab0 tx=0x7f57b403f040 comp rx=0 tx=0).stop 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 -- 172.21.15.102:0/1265283478 shutdown_connections 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55d2d1728bf0 0x55d2d172b010 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d16b2230 0x55d2d16abf20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d16a7b60 0x55d2d16ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 --2- 172.21.15.102:0/1265283478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2d16a7240 0x55d2d16ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 -- 172.21.15.102:0/1265283478 >> 172.21.15.102:0/1265283478 conn(0x55d2d16a3560 msgr2=0x55d2d16a1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f579f7fe640 0 client.15990 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 -- 172.21.15.102:0/1265283478 shutdown_connections 2024-04-24T23:20:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 1 -- 172.21.15.102:0/1265283478 wait complete. 2024-04-24T23:20:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.276+0000 7f57c95a8740 20 client.15990 trim_cache size 0 max 16384 2024-04-24T23:20:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:20:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:20:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:20:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 -- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f57a80a2da0 msgr2=0x7f57a80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f57a80a2da0 0x7f57a80a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f57b80ccdb0 tx=0x7f57b806aab0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 -- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0106630 msgr2=0x7f57c0197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0106630 0x7f57c0197590 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f57b800eaa0 tx=0x7f57b800ef60 comp rx=0 tx=0).stop 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 -- 172.21.15.102:0/1335019009 shutdown_connections 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f57a80a2da0 0x7f57a80a5260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57c0106630 0x7f57c0197590 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57c0101030 0x7f57c0197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 --2- 172.21.15.102:0/1335019009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c0100670 0x7f57c0196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 -- 172.21.15.102:0/1335019009 >> 172.21.15.102:0/1335019009 conn(0x7f57c00777c0 msgr2=0x7f57c0103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 -- 172.21.15.102:0/1335019009 shutdown_connections 2024-04-24T23:20:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:18.277+0000 7f57c67c4640 1 -- 172.21.15.102:0/1335019009 wait complete. 2024-04-24T23:20:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:19 smithi183 ceph-mon[33137]: pgmap v223: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-04-24T23:20:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:19 smithi102 ceph-mon[29216]: pgmap v223: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-04-24T23:20:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:19 smithi161 ceph-mon[32709]: pgmap v223: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-04-24T23:20:21.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:21 smithi183 ceph-mon[33137]: pgmap v224: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-04-24T23:20:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:21 smithi102 ceph-mon[29216]: pgmap v224: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-04-24T23:20:21.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:21 smithi161 ceph-mon[32709]: pgmap v224: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 MiB/s wr, 18 op/s 2024-04-24T23:20:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:23 smithi183 ceph-mon[33137]: pgmap v225: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:20:23.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:23 smithi102 ceph-mon[29216]: pgmap v225: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:20:23.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:23 smithi161 ceph-mon[32709]: pgmap v225: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 16 op/s 2024-04-24T23:20:25.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:25 smithi183 ceph-mon[33137]: pgmap v226: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:20:25.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:25 smithi102 ceph-mon[29216]: pgmap v226: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:20:25.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:25 smithi161 ceph-mon[32709]: pgmap v226: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:20:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:27 smithi183 ceph-mon[33137]: pgmap v227: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:20:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:27 smithi102 ceph-mon[29216]: pgmap v227: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:20:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:27 smithi161 ceph-mon[32709]: pgmap v227: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:20:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:29 smithi183 ceph-mon[33137]: pgmap v228: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:29 smithi102 ceph-mon[29216]: pgmap v228: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:29 smithi161 ceph-mon[32709]: pgmap v228: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 -- 172.21.15.102:0/3454522322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 msgr2=0x7f40fc10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 --2- 172.21.15.102:0/3454522322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 0x7f40fc10d4f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009cd0 tx=0x7f40f402a8e0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 -- 172.21.15.102:0/3454522322 shutdown_connections 2024-04-24T23:20:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 --2- 172.21.15.102:0/3454522322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 0x7f40fc10d4f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 --2- 172.21.15.102:0/3454522322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40fc100670 0x7f40fc100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 --2- 172.21.15.102:0/3454522322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40fc106630 0x7f40fc106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 -- 172.21.15.102:0/3454522322 >> 172.21.15.102:0/3454522322 conn(0x7f40fc0777c0 msgr2=0x7f40fc077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 -- 172.21.15.102:0/3454522322 shutdown_connections 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 -- 172.21.15.102:0/3454522322 wait complete. 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 Processor -- start 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.392+0000 7f4104276640 1 -- start start 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4104276640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40fc100670 0x7f40fc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4104276640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 0x7f40fc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4104276640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40fc106630 0x7f40fc19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4104276640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1108f0 con 0x7f40fc101030 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4104276640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc110770 con 0x7f40fc106630 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4104276640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc110a70 con 0x7f40fc100670 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4101811640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 0x7f40fc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4101811640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 0x7f40fc198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53662/0 (socket says 172.21.15.102:53662) 2024-04-24T23:20:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4101811640 1 -- 172.21.15.102:0/2442110286 learned_addr learned my addr 172.21.15.102:0/2442110286 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4102813640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40fc106630 0x7f40fc19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4102012640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40fc100670 0x7f40fc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4101811640 1 -- 172.21.15.102:0/2442110286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40fc100670 msgr2=0x7f40fc197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4101811640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40fc100670 0x7f40fc197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4101811640 1 -- 172.21.15.102:0/2442110286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40fc106630 msgr2=0x7f40fc19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4101811640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40fc106630 0x7f40fc19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4101811640 1 -- 172.21.15.102:0/2442110286 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc19cf20 con 0x7f40fc101030 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4101811640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 0x7f40fc198470 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00d950 tx=0x7f40ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f40f37fe640 1 -- 172.21.15.102:0/2442110286 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec017070 con 0x7f40fc101030 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f40f37fe640 1 -- 172.21.15.102:0/2442110286 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40ec009c30 con 0x7f40fc101030 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4104276640 1 -- 172.21.15.102:0/2442110286 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc19d210 con 0x7f40fc101030 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f40f37fe640 1 -- 172.21.15.102:0/2442110286 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00f620 con 0x7f40fc101030 2024-04-24T23:20:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.393+0000 7f4104276640 1 -- 172.21.15.102:0/2442110286 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc1a4aa0 con 0x7f40fc101030 2024-04-24T23:20:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.396+0000 7f40f37fe640 1 -- 172.21.15.102:0/2442110286 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f40ec0109a0 con 0x7f40fc101030 2024-04-24T23:20:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.396+0000 7f4106f84740 1 Processor -- start 2024-04-24T23:20:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4106f84740 1 -- start start 2024-04-24T23:20:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f40f37fe640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f40e80a2da0 0x7f40e80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4106f84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 0x55eb391b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4106f84740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 0x55eb391bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4106f84740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4106f84740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55eb39119cb0 con 0x55eb391c0230 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4106f84740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb390c25b0 con 0x55eb391b5b60 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4106f84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb39115c70 con 0x55eb391b5240 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102813640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f40e80a2da0 0x7f40e80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102813640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102813640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53666/0 (socket says 172.21.15.102:53666) 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f40f37fe640 1 -- 172.21.15.102:0/2442110286 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7f40ec0cf170 con 0x7f40fc101030 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102813640 1 -- 172.21.15.102:0/2376141357 learned_addr learned my addr 172.21.15.102:0/2376141357 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102012640 1 --2- 172.21.15.102:0/2376141357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 0x55eb391b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4101811640 1 --2- 172.21.15.102:0/2376141357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 0x55eb391bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102813640 1 -- 172.21.15.102:0/2376141357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 msgr2=0x55eb391b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102813640 1 --2- 172.21.15.102:0/2376141357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 0x55eb391b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102813640 1 -- 172.21.15.102:0/2376141357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 msgr2=0x55eb391bfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102813640 1 --2- 172.21.15.102:0/2376141357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 0x55eb391bfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.397+0000 7f4102813640 1 -- 172.21.15.102:0/2376141357 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb38e2e210 con 0x55eb391c0230 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.398+0000 7f4102813640 1 --2- 172.21.15.102:0/2376141357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c2620 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f40f4004af0 tx=0x7f40f4031a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464757b8d2841384 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.398+0000 7f4102813640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f40e80a2da0 0x7f40e80a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009a60 tx=0x7f40f4042000 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.398+0000 7f40f0bfe640 1 -- 172.21.15.102:0/2376141357 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f404e070 con 0x55eb391c0230 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.398+0000 7f40f0bfe640 1 -- 172.21.15.102:0/2376141357 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40f4036040 con 0x55eb391c0230 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.398+0000 7f40f0bfe640 1 -- 172.21.15.102:0/2376141357 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40307c0 con 0x55eb391c0230 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 -- 172.21.15.102:0/2376141357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 msgr2=0x55eb391c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 --2- 172.21.15.102:0/2376141357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c2620 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f40f4004af0 tx=0x7f40f4031a80 comp rx=0 tx=0).stop 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 -- 172.21.15.102:0/2376141357 shutdown_connections 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 --2- 172.21.15.102:0/2376141357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c2620 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 --2- 172.21.15.102:0/2376141357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 0x55eb391bfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 --2- 172.21.15.102:0/2376141357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 0x55eb391b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 -- 172.21.15.102:0/2376141357 >> 172.21.15.102:0/2376141357 conn(0x55eb391b1560 msgr2=0x55eb391b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 -- 172.21.15.102:0/2376141357 shutdown_connections 2024-04-24T23:20:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 -- 172.21.15.102:0/2376141357 wait complete. 2024-04-24T23:20:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 -- start start 2024-04-24T23:20:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.399+0000 7f4106f84740 1 Processor -- start 2024-04-24T23:20:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4106f84740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4106f84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 0x55eb391bd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4106f84740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391c0230 0x55eb391bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4106f84740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55eb391c2fc0 con 0x55eb391b5240 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4106f84740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb390d5b50 con 0x55eb391c0230 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4106f84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb391c3140 con 0x55eb391b5b60 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4106f84740 10 client.0 _ll_register_callbacks cb 0x55eb39110ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102012640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102012640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53674/0 (socket says 172.21.15.102:53674) 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102012640 1 -- 172.21.15.102:0/2942294210 learned_addr learned my addr 172.21.15.102:0/2942294210 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4101811640 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 0x55eb391bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102813640 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391c0230 0x55eb391bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102012640 1 -- 172.21.15.102:0/2942294210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 msgr2=0x55eb391bd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102012640 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 0x55eb391bd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102012640 1 -- 172.21.15.102:0/2942294210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391c0230 msgr2=0x55eb391bdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102012640 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391c0230 0x55eb391bdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102012640 1 -- 172.21.15.102:0/2942294210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb38efa620 con 0x55eb391b5240 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f4102012640 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bd1f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f40f8012730 tx=0x7f40f8012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4584a9b8fb340862 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f40e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.400+0000 7f40e5ffb640 1 -- 172.21.15.102:0/2942294210 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011b50 con 0x55eb391b5240 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f40e5ffb640 1 -- 172.21.15.102:0/2942294210 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7f40f8011cf0 con 0x55eb391b5240 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4106f84740 1 -- 172.21.15.102:0/2942294210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f4036040 con 0x55eb391b5240 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f40e5ffb640 1 -- 172.21.15.102:0/2942294210 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40f8004020 con 0x55eb391b5240 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4106f84740 10 client.16002 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4106f84740 1 -- 172.21.15.102:0/2942294210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb391cf260 con 0x55eb391b5240 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f40e5ffb640 1 -- 172.21.15.102:0/2942294210 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40f4036040 con 0x55eb391b5240 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f40e5ffb640 1 -- 172.21.15.102:0/2942294210 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f40f801c470 con 0x55eb391b5240 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4106f84740 10 client.16002 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4106f84740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4106f84740 20 client.16002 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4106f84740 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb391d0770 0x55eb391d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4106f84740 4 client.16002 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4106f84740 1 -- 172.21.15.102:0/2942294210 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb391271f0 con 0x55eb391d0770 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.401+0000 7f4101811640 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb391d0770 0x55eb391d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.402+0000 7f40e5ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.402+0000 7f4101811640 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb391d0770 0x55eb391d2b60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d286b33848a92045 server_cookie=7f356f651d74eea in_seq=0 out_seq=0 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.402+0000 7f40e5ffb640 1 -- 172.21.15.102:0/2942294210 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb391271f0 con 0x55eb391d0770 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.402+0000 7f40e5ffb640 10 client.16002 handle_command_reply: tid=0 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 client.16002 shutdown 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 -- 172.21.15.102:0/2942294210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 msgr2=0x55eb391bd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bd1f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f40f8012730 tx=0x7f40f8012bf0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 -- 172.21.15.102:0/2942294210 shutdown_connections 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391c0230 0x55eb391bdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 0x55eb391bd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb391d0770 0x55eb391d2b60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 --2- 172.21.15.102:0/2942294210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bd1f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f40e5ffb640 0 client.16002 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 -- 172.21.15.102:0/2942294210 >> 172.21.15.102:0/2942294210 conn(0x55eb391b1560 msgr2=0x55eb391b7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 -- 172.21.15.102:0/2942294210 shutdown_connections 2024-04-24T23:20:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 1 -- 172.21.15.102:0/2942294210 wait complete. 2024-04-24T23:20:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.403+0000 7f4106f84740 20 client.16002 trim_cache size 0 max 16384 2024-04-24T23:20:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.422+0000 7f4106f84740 1 Processor -- start 2024-04-24T23:20:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4106f84740 1 -- start start 2024-04-24T23:20:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4106f84740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4106f84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 0x55eb391cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4106f84740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391c0230 0x55eb391d8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4106f84740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1afd10 con 0x55eb391b5240 2024-04-24T23:20:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4106f84740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1ad5e0 con 0x55eb391c0230 2024-04-24T23:20:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4106f84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1ab450 con 0x55eb391b5b60 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4102012640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4102012640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53686/0 (socket says 172.21.15.102:53686) 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4102012640 1 -- 172.21.15.102:0/1014400170 learned_addr learned my addr 172.21.15.102:0/1014400170 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4101811640 1 --2- 172.21.15.102:0/1014400170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 0x55eb391cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4102012640 1 -- 172.21.15.102:0/1014400170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 msgr2=0x55eb391cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4102012640 1 --2- 172.21.15.102:0/1014400170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 0x55eb391cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4102012640 1 -- 172.21.15.102:0/1014400170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391c0230 msgr2=0x55eb391d8400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.423+0000 7f4102012640 1 --2- 172.21.15.102:0/1014400170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391c0230 0x55eb391d8400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.424+0000 7f4102012640 1 -- 172.21.15.102:0/1014400170 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb391b5990 con 0x55eb391b5240 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.424+0000 7f4102012640 1 --2- 172.21.15.102:0/1014400170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bf920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f40f8001eb0 tx=0x7f40f801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c8963d50dfffff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.424+0000 7f40e7fff640 1 -- 172.21.15.102:0/1014400170 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8028040 con 0x55eb391b5240 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.424+0000 7f40e7fff640 1 -- 172.21.15.102:0/1014400170 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40f801cad0 con 0x55eb391b5240 2024-04-24T23:20:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.424+0000 7f40e7fff640 1 -- 172.21.15.102:0/1014400170 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8005070 con 0x55eb391b5240 2024-04-24T23:20:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.425+0000 7f4106f84740 1 -- 172.21.15.102:0/1014400170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 msgr2=0x55eb391bf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.425+0000 7f4106f84740 1 --2- 172.21.15.102:0/1014400170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bf920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f40f8001eb0 tx=0x7f40f801c680 comp rx=0 tx=0).stop 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.425+0000 7f4106f84740 1 -- 172.21.15.102:0/1014400170 shutdown_connections 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.425+0000 7f4106f84740 1 --2- 172.21.15.102:0/1014400170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391c0230 0x55eb391d8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.425+0000 7f4106f84740 1 --2- 172.21.15.102:0/1014400170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5b60 0x55eb391cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.425+0000 7f4106f84740 1 --2- 172.21.15.102:0/1014400170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391b5240 0x55eb391bf920 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.425+0000 7f4106f84740 1 -- 172.21.15.102:0/1014400170 >> 172.21.15.102:0/1014400170 conn(0x55eb391b1560 msgr2=0x55eb391b84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.425+0000 7f4106f84740 1 -- 172.21.15.102:0/1014400170 shutdown_connections 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.425+0000 7f4106f84740 1 -- 172.21.15.102:0/1014400170 wait complete. 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4106f84740 1 -- start start 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4106f84740 1 Processor -- start 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4106f84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 0x55eb391c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4106f84740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 0x55eb391c3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4106f84740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4106f84740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1abaa0 con 0x55eb391c0230 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4106f84740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1acf10 con 0x55eb391b5b60 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4106f84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1ac170 con 0x55eb391b5240 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4106f84740 10 client.0 _ll_register_callbacks cb 0x7f40f80096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4102813640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4102813640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53702/0 (socket says 172.21.15.102:53702) 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4102813640 1 -- 172.21.15.102:0/1142334948 learned_addr learned my addr 172.21.15.102:0/1142334948 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4102012640 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 0x55eb391c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.426+0000 7f4101811640 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 0x55eb391c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f4102813640 1 -- 172.21.15.102:0/1142334948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 msgr2=0x55eb391c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f4102813640 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 0x55eb391c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f4102813640 1 -- 172.21.15.102:0/1142334948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 msgr2=0x55eb391c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f4102813640 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 0x55eb391c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f4102813640 1 -- 172.21.15.102:0/1142334948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb391c4430 con 0x55eb391c0230 2024-04-24T23:20:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f4102813640 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c3e70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f40f4003e70 tx=0x7f40f4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8c27a1e4ccbe4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f40e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f40e5ffb640 1 -- 172.21.15.102:0/1142334948 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f404e040 con 0x55eb391c0230 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f40e5ffb640 1 -- 172.21.15.102:0/1142334948 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7f40f402e040 con 0x55eb391c0230 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f4106f84740 1 -- 172.21.15.102:0/1142334948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f801cad0 con 0x55eb391c0230 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f40e5ffb640 1 -- 172.21.15.102:0/1142334948 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40f402af20 con 0x55eb391c0230 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f4106f84740 10 client.16008 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.427+0000 7f4106f84740 1 -- 172.21.15.102:0/1142334948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb39236a30 con 0x55eb391c0230 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.428+0000 7f40e5ffb640 1 -- 172.21.15.102:0/1142334948 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40f801cad0 con 0x55eb391c0230 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.428+0000 7f40e5ffb640 1 -- 172.21.15.102:0/1142334948 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f40f4030070 con 0x55eb391c0230 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.428+0000 7f4106f84740 10 client.16008 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.428+0000 7f4106f84740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.428+0000 7f4106f84740 20 client.16008 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.428+0000 7f4106f84740 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb39236d80 0x55eb392391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.428+0000 7f4106f84740 4 client.16008 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:20:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.428+0000 7f4106f84740 1 -- 172.21.15.102:0/1142334948 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb39121540 con 0x55eb39236d80 2024-04-24T23:20:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.428+0000 7f4102012640 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb39236d80 0x55eb392391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.429+0000 7f4102012640 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb39236d80 0x55eb392391a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59c14b7bdbba133 server_cookie=60273b6a5ec01583 in_seq=0 out_seq=0 2024-04-24T23:20:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.429+0000 7f40e5ffb640 10 client.16008 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f40e5ffb640 1 -- 172.21.15.102:0/1142334948 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb39121540 con 0x55eb39236d80 2024-04-24T23:20:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f40e5ffb640 10 client.16008 handle_command_reply: tid=0 2024-04-24T23:20:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f4106f84740 1 client.16008 shutdown 2024-04-24T23:20:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f4106f84740 1 -- 172.21.15.102:0/1142334948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 msgr2=0x55eb391c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f4106f84740 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c3e70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f40f4003e70 tx=0x7f40f4009800 comp rx=0 tx=0).stop 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f4106f84740 1 -- 172.21.15.102:0/1142334948 shutdown_connections 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f4106f84740 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb39236d80 0x55eb392391a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f4106f84740 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb391c0230 0x55eb391c3e70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f4106f84740 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb391b5b60 0x55eb391c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.436+0000 7f4106f84740 1 --2- 172.21.15.102:0/1142334948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb391b5240 0x55eb391c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4106f84740 1 -- 172.21.15.102:0/1142334948 >> 172.21.15.102:0/1142334948 conn(0x55eb391b1560 msgr2=0x55eb391afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f40e5ffb640 0 client.16008 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4106f84740 1 -- 172.21.15.102:0/1142334948 shutdown_connections 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4106f84740 1 -- 172.21.15.102:0/1142334948 wait complete. 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4106f84740 20 client.16008 trim_cache size 0 max 16384 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:20:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:20:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:20:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:20:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 -- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f40e80a2da0 msgr2=0x7f40e80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f40e80a2da0 0x7f40e80a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009a60 tx=0x7f40f4042000 comp rx=0 tx=0).stop 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 -- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 msgr2=0x7f40fc198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 0x7f40fc198470 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00d950 tx=0x7f40ec00de10 comp rx=0 tx=0).stop 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 -- 172.21.15.102:0/2442110286 shutdown_connections 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40fc106630 0x7f40fc19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f40e80a2da0 0x7f40e80a5260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40fc101030 0x7f40fc198470 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 --2- 172.21.15.102:0/2442110286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40fc100670 0x7f40fc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 -- 172.21.15.102:0/2442110286 >> 172.21.15.102:0/2442110286 conn(0x7f40fc0777c0 msgr2=0x7f40fc069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 -- 172.21.15.102:0/2442110286 shutdown_connections 2024-04-24T23:20:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:31.437+0000 7f4104276640 1 -- 172.21.15.102:0/2442110286 wait complete. 2024-04-24T23:20:31.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:31 smithi183 ceph-mon[33137]: pgmap v229: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:20:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:31 smithi102 ceph-mon[29216]: pgmap v229: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:20:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:31 smithi161 ceph-mon[32709]: pgmap v229: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:20:33.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:33 smithi183 ceph-mon[33137]: pgmap v230: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:20:33.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:33 smithi102 ceph-mon[29216]: pgmap v230: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:20:33.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:33 smithi161 ceph-mon[32709]: pgmap v230: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-04-24T23:20:35.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:35 smithi183 ceph-mon[33137]: pgmap v231: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:20:35.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:35 smithi102 ceph-mon[29216]: pgmap v231: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:20:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:35 smithi161 ceph-mon[32709]: pgmap v231: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:20:37.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:37 smithi183 ceph-mon[33137]: pgmap v232: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:37.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:37 smithi102 ceph-mon[29216]: pgmap v232: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:37 smithi161 ceph-mon[32709]: pgmap v232: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:39.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:39 smithi183 ceph-mon[33137]: pgmap v233: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:39.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:39 smithi102 ceph-mon[29216]: pgmap v233: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:39.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:39 smithi161 ceph-mon[32709]: pgmap v233: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:41 smithi183 ceph-mon[33137]: pgmap v234: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:41.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:41 smithi102 ceph-mon[29216]: pgmap v234: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:41 smithi161 ceph-mon[32709]: pgmap v234: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:43.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:43 smithi183 ceph-mon[33137]: pgmap v235: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:43.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:43 smithi102 ceph-mon[29216]: pgmap v235: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:43.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:43 smithi161 ceph-mon[32709]: pgmap v235: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.741+0000 7fe9acc15640 1 -- 172.21.15.102:0/720661463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 msgr2=0x7fe9a8101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.741+0000 7fe9acc15640 1 --2- 172.21.15.102:0/720661463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 0x7fe9a8101ba0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009930 tx=0x7fe99c02a420 comp rx=0 tx=0).stop 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.741+0000 7fe9acc15640 1 -- 172.21.15.102:0/720661463 shutdown_connections 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.741+0000 7fe9acc15640 1 --2- 172.21.15.102:0/720661463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9a81020e0 0x7fe9a810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.741+0000 7fe9acc15640 1 --2- 172.21.15.102:0/720661463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 0x7fe9a8101ba0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.741+0000 7fe9acc15640 1 --2- 172.21.15.102:0/720661463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a8107730 0x7fe9a8107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.741+0000 7fe9acc15640 1 -- 172.21.15.102:0/720661463 >> 172.21.15.102:0/720661463 conn(0x7fe9a80fd1f0 msgr2=0x7fe9a80ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.741+0000 7fe9acc15640 1 -- 172.21.15.102:0/720661463 shutdown_connections 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.741+0000 7fe9acc15640 1 -- 172.21.15.102:0/720661463 wait complete. 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9acc15640 1 Processor -- start 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9acc15640 1 -- start start 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9acc15640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 0x7fe9a819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9acc15640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9a81020e0 0x7fe9a819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9acc15640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a8107730 0x7fe9a819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9acc15640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8111a40 con 0x7fe9a8101720 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9acc15640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81118c0 con 0x7fe9a8107730 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9acc15640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8111bc0 con 0x7fe9a81020e0 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 0x7fe9a819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 0x7fe9a819a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43968/0 (socket says 172.21.15.102:43968) 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a659c640 1 -- 172.21.15.102:0/189163283 learned_addr learned my addr 172.21.15.102:0/189163283 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a5d9b640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9a81020e0 0x7fe9a819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a8107730 0x7fe9a819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a659c640 1 -- 172.21.15.102:0/189163283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9a81020e0 msgr2=0x7fe9a819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a659c640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9a81020e0 0x7fe9a819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a659c640 1 -- 172.21.15.102:0/189163283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a8107730 msgr2=0x7fe9a819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a659c640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a8107730 0x7fe9a819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.742+0000 7fe9a659c640 1 -- 172.21.15.102:0/189163283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a819f070 con 0x7fe9a8101720 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.743+0000 7fe9a659c640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 0x7fe9a819a080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe99800e9f0 tx=0x7fe99800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.743+0000 7fe9977fe640 1 -- 172.21.15.102:0/189163283 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99800cd80 con 0x7fe9a8101720 2024-04-24T23:20:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.743+0000 7fe9acc15640 1 -- 172.21.15.102:0/189163283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a819f360 con 0x7fe9a8101720 2024-04-24T23:20:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.743+0000 7fe9977fe640 1 -- 172.21.15.102:0/189163283 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9980108e0 con 0x7fe9a8101720 2024-04-24T23:20:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.743+0000 7fe9977fe640 1 -- 172.21.15.102:0/189163283 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998018920 con 0x7fe9a8101720 2024-04-24T23:20:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.743+0000 7fe9acc15640 1 -- 172.21.15.102:0/189163283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a81a6bf0 con 0x7fe9a8101720 2024-04-24T23:20:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.745+0000 7fe9977fe640 1 -- 172.21.15.102:0/189163283 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe998010a90 con 0x7fe9a8101720 2024-04-24T23:20:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.746+0000 7fe9977fe640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe9900a2da0 0x7fe9900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.746+0000 7fe9a5d9b640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe9900a2da0 0x7fe9900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.746+0000 7fe9977fe640 1 -- 172.21.15.102:0/189163283 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7fe9980cf7d0 con 0x7fe9a8101720 2024-04-24T23:20:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.746+0000 7fe9a5d9b640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe9900a2da0 0x7fe9900a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe99c031440 tx=0x7fe99c005a20 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9af9ff740 1 Processor -- start 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9af9ff740 1 -- start start 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec655620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 0x5647ec65fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec660230 0x5647ec662620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5647ec5b9cb0 con 0x5647ec655240 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5647ec5625b0 con 0x5647ec655b60 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5647ec5b5c70 con 0x5647ec660230 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec655620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec655620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43970/0 (socket says 172.21.15.102:43970) 2024-04-24T23:20:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9a659c640 1 -- 172.21.15.102:0/1258353573 learned_addr learned my addr 172.21.15.102:0/1258353573 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.756+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec660230 0x5647ec662620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe9a5d9b640 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 0x5647ec65fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe9a659c640 1 -- 172.21.15.102:0/1258353573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec660230 msgr2=0x5647ec662620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe9a659c640 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec660230 0x5647ec662620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe9a659c640 1 -- 172.21.15.102:0/1258353573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 msgr2=0x5647ec65fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe9a659c640 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 0x5647ec65fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe9a659c640 1 -- 172.21.15.102:0/1258353573 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647ec2ce210 con 0x5647ec655240 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe9a5d9b640 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 0x5647ec65fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe9a659c640 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec655620 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe998009380 tx=0x7fe99809b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f6886c7f13a22e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe994ff9640 1 -- 172.21.15.102:0/1258353573 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9980a4920 con 0x5647ec655240 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe994ff9640 1 -- 172.21.15.102:0/1258353573 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe998099fb0 con 0x5647ec655240 2024-04-24T23:20:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.757+0000 7fe994ff9640 1 -- 172.21.15.102:0/1258353573 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99809b3f0 con 0x5647ec655240 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 -- 172.21.15.102:0/1258353573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 msgr2=0x5647ec655620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec655620 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe998009380 tx=0x7fe99809b920 comp rx=0 tx=0).stop 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 -- 172.21.15.102:0/1258353573 shutdown_connections 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec660230 0x5647ec662620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 0x5647ec65fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/1258353573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec655620 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 -- 172.21.15.102:0/1258353573 >> 172.21.15.102:0/1258353573 conn(0x5647ec651560 msgr2=0x5647ec651940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 -- 172.21.15.102:0/1258353573 shutdown_connections 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 -- 172.21.15.102:0/1258353573 wait complete. 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 -- start start 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.758+0000 7fe9af9ff740 1 Processor -- start 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655240 0x5647ec65d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 0x5647ec65d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec65dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5647ec662fc0 con 0x5647ec660230 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5647ec575b50 con 0x5647ec655b60 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5647ec663140 con 0x5647ec655240 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9af9ff740 10 client.0 _ll_register_callbacks cb 0x5647ec5b0ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a6d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec65dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655240 0x5647ec65d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a6d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec65dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43978/0 (socket says 172.21.15.102:43978) 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655240 0x5647ec65d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:50340/0 (socket says 172.21.15.102:50340) 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a6d9d640 1 -- 172.21.15.102:0/3495795081 learned_addr learned my addr 172.21.15.102:0/3495795081 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a5d9b640 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 0x5647ec65d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a6d9d640 1 -- 172.21.15.102:0/3495795081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655240 msgr2=0x5647ec65d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655240 0x5647ec65d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a6d9d640 1 -- 172.21.15.102:0/3495795081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 msgr2=0x5647ec65d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 0x5647ec65d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a6d9d640 1 -- 172.21.15.102:0/3495795081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647ec39a620 con 0x5647ec660230 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a659c640 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655240 0x5647ec65d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec65dbb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0004070 tx=0x7fe9a0004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475f1583c97fd251 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe98dffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe98dffb640 1 -- 172.21.15.102:0/3495795081 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0004b20 con 0x5647ec660230 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe98dffb640 1 -- 172.21.15.102:0/3495795081 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7fe9a0004cc0 con 0x5647ec660230 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.759+0000 7fe9af9ff740 1 -- 172.21.15.102:0/3495795081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe998099fb0 con 0x5647ec660230 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe98dffb640 1 -- 172.21.15.102:0/3495795081 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9a0011920 con 0x5647ec660230 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe9af9ff740 10 client.16020 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe9af9ff740 1 -- 172.21.15.102:0/3495795081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647ec66f100 con 0x5647ec660230 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe98dffb640 1 -- 172.21.15.102:0/3495795081 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe998099fb0 con 0x5647ec660230 2024-04-24T23:20:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe98dffb640 1 -- 172.21.15.102:0/3495795081 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe9a001bc40 con 0x5647ec660230 2024-04-24T23:20:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe9af9ff740 10 client.16020 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe9af9ff740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe9af9ff740 20 client.16020 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5647ec6705c0 0x5647ec6729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe9af9ff740 4 client.16020 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe9af9ff740 1 -- 172.21.15.102:0/3495795081 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647ec5c71f0 con 0x5647ec6705c0 2024-04-24T23:20:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.760+0000 7fe9a659c640 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5647ec6705c0 0x5647ec6729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.761+0000 7fe98dffb640 10 client.16020 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.761+0000 7fe9a659c640 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5647ec6705c0 0x5647ec6729b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6650f3f3003d8e70 server_cookie=d0652b72827d8cd9 in_seq=0 out_seq=0 2024-04-24T23:20:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.761+0000 7fe98dffb640 1 -- 172.21.15.102:0/3495795081 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647ec5c71f0 con 0x5647ec6705c0 2024-04-24T23:20:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.761+0000 7fe98dffb640 10 client.16020 handle_command_reply: tid=0 2024-04-24T23:20:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.761+0000 7fe9af9ff740 1 client.16020 shutdown 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 -- 172.21.15.102:0/3495795081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 msgr2=0x5647ec65dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec65dbb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0004070 tx=0x7fe9a0004530 comp rx=0 tx=0).stop 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 -- 172.21.15.102:0/3495795081 shutdown_connections 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5647ec6705c0 0x5647ec6729b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec65dbb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655b60 0x5647ec65d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/3495795081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655240 0x5647ec65d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 -- 172.21.15.102:0/3495795081 >> 172.21.15.102:0/3495795081 conn(0x5647ec651560 msgr2=0x5647ec657240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe98dffb640 0 client.16020 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 -- 172.21.15.102:0/3495795081 shutdown_connections 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 1 -- 172.21.15.102:0/3495795081 wait complete. 2024-04-24T23:20:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.762+0000 7fe9af9ff740 20 client.16020 trim_cache size 0 max 16384 2024-04-24T23:20:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.784+0000 7fe9af9ff740 1 Processor -- start 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.784+0000 7fe9af9ff740 1 -- start start 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec66be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 0x5647ec6579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec660230 0x5647ec6782c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b1fc0 con 0x5647ec655240 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81af890 con 0x5647ec660230 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81ad700 con 0x5647ec655b60 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec66be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec66be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43986/0 (socket says 172.21.15.102:43986) 2024-04-24T23:20:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a659c640 1 -- 172.21.15.102:0/2518434500 learned_addr learned my addr 172.21.15.102:0/2518434500 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a5d9b640 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 0x5647ec6579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec660230 0x5647ec6782c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a659c640 1 -- 172.21.15.102:0/2518434500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 msgr2=0x5647ec6579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a659c640 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 0x5647ec6579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a659c640 1 -- 172.21.15.102:0/2518434500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec660230 msgr2=0x5647ec6782c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a659c640 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec660230 0x5647ec6782c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a659c640 1 -- 172.21.15.102:0/2518434500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647ec655040 con 0x5647ec655240 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec660230 0x5647ec6782c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe9a659c640 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec66be80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe99800c6a0 tx=0x7fe9980a01e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f6a7cf112bc704 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe98ffff640 1 -- 172.21.15.102:0/2518434500 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9980b8070 con 0x5647ec655240 2024-04-24T23:20:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe98ffff640 1 -- 172.21.15.102:0/2518434500 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe998098320 con 0x5647ec655240 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.785+0000 7fe98ffff640 1 -- 172.21.15.102:0/2518434500 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998018ac0 con 0x5647ec655240 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 -- 172.21.15.102:0/2518434500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 msgr2=0x5647ec66be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec66be80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe99800c6a0 tx=0x7fe9980a01e0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 -- 172.21.15.102:0/2518434500 shutdown_connections 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec660230 0x5647ec6782c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 0x5647ec6579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/2518434500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec655240 0x5647ec66be80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 -- 172.21.15.102:0/2518434500 >> 172.21.15.102:0/2518434500 conn(0x5647ec651560 msgr2=0x5647ec6505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 -- 172.21.15.102:0/2518434500 shutdown_connections 2024-04-24T23:20:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 -- 172.21.15.102:0/2518434500 wait complete. 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 -- start start 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.786+0000 7fe9af9ff740 1 Processor -- start 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655240 0x5647ec6685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 0x5647ec659a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9af9ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec659f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81add50 con 0x5647ec660230 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81af1c0 con 0x5647ec655240 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9af9ff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81ae420 con 0x5647ec655b60 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9af9ff740 10 client.0 _ll_register_callbacks cb 0x7fe9a0009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a6d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec659f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a6d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec659f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43992/0 (socket says 172.21.15.102:43992) 2024-04-24T23:20:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a6d9d640 1 -- 172.21.15.102:0/474836340 learned_addr learned my addr 172.21.15.102:0/474836340 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a5d9b640 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 0x5647ec659a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a659c640 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655240 0x5647ec6685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a6d9d640 1 -- 172.21.15.102:0/474836340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 msgr2=0x5647ec659a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 0x5647ec659a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a6d9d640 1 -- 172.21.15.102:0/474836340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655240 msgr2=0x5647ec6685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655240 0x5647ec6685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a6d9d640 1 -- 172.21.15.102:0/474836340 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647ec65a580 con 0x5647ec660230 2024-04-24T23:20:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe9a6d9d640 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec659f40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0004040 tx=0x7fe9a0003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55e1e6c76ca0e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe98dffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.787+0000 7fe98dffb640 1 -- 172.21.15.102:0/474836340 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0011d90 con 0x5647ec660230 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe98dffb640 1 -- 172.21.15.102:0/474836340 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7fe9a0017040 con 0x5647ec660230 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9af9ff740 1 -- 172.21.15.102:0/474836340 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe998098320 con 0x5647ec660230 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe98dffb640 1 -- 172.21.15.102:0/474836340 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9a0010a90 con 0x5647ec660230 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9af9ff740 10 client.16026 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9af9ff740 1 -- 172.21.15.102:0/474836340 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647ec6d6960 con 0x5647ec660230 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe98dffb640 1 -- 172.21.15.102:0/474836340 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe998098320 con 0x5647ec660230 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe98dffb640 1 -- 172.21.15.102:0/474836340 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe9a002c710 con 0x5647ec660230 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9af9ff740 10 client.16026 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9af9ff740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9af9ff740 20 client.16026 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5647ec6d7110 0x5647ec6d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9af9ff740 4 client.16026 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:20:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9af9ff740 1 -- 172.21.15.102:0/474836340 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647ec5c1540 con 0x5647ec6d7110 2024-04-24T23:20:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.788+0000 7fe9a659c640 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5647ec6d7110 0x5647ec6d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.789+0000 7fe9a659c640 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5647ec6d7110 0x5647ec6d9530 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1945eeeb1779ac81 server_cookie=1ed98cf2be03788a in_seq=0 out_seq=0 2024-04-24T23:20:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.789+0000 7fe98dffb640 10 client.16026 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe98dffb640 1 -- 172.21.15.102:0/474836340 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647ec5c1540 con 0x5647ec6d7110 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe98dffb640 10 client.16026 handle_command_reply: tid=0 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe9af9ff740 1 client.16026 shutdown 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe9af9ff740 1 -- 172.21.15.102:0/474836340 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 msgr2=0x5647ec659f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec659f40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0004040 tx=0x7fe9a0003f20 comp rx=0 tx=0).stop 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe9af9ff740 1 -- 172.21.15.102:0/474836340 shutdown_connections 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5647ec6d7110 0x5647ec6d9530 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe98dffb640 0 client.16026 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647ec660230 0x5647ec659f40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647ec655b60 0x5647ec659a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe9af9ff740 1 --2- 172.21.15.102:0/474836340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647ec655240 0x5647ec6685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.796+0000 7fe9af9ff740 1 -- 172.21.15.102:0/474836340 >> 172.21.15.102:0/474836340 conn(0x5647ec651560 msgr2=0x5647ec65d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9af9ff740 1 -- 172.21.15.102:0/474836340 shutdown_connections 2024-04-24T23:20:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9af9ff740 1 -- 172.21.15.102:0/474836340 wait complete. 2024-04-24T23:20:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9af9ff740 20 client.16026 trim_cache size 0 max 16384 2024-04-24T23:20:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:20:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:20:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:20:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:20:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 -- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe9900a2da0 msgr2=0x7fe9900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe9900a2da0 0x7fe9900a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe99c031440 tx=0x7fe99c005a20 comp rx=0 tx=0).stop 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 -- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 msgr2=0x7fe9a819a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 0x7fe9a819a080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe99800e9f0 tx=0x7fe99800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 -- 172.21.15.102:0/189163283 shutdown_connections 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a8107730 0x7fe9a819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9a81020e0 0x7fe9a819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe9900a2da0 0x7fe9900a5260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 --2- 172.21.15.102:0/189163283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9a8101720 0x7fe9a819a080 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 -- 172.21.15.102:0/189163283 >> 172.21.15.102:0/189163283 conn(0x7fe9a80fd1f0 msgr2=0x7fe9a810c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 -- 172.21.15.102:0/189163283 shutdown_connections 2024-04-24T23:20:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:44.797+0000 7fe9acc15640 1 -- 172.21.15.102:0/189163283 wait complete. 2024-04-24T23:20:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:45 smithi183 ceph-mon[33137]: pgmap v236: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:45.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:45 smithi102 ceph-mon[29216]: pgmap v236: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:45.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:45 smithi161 ceph-mon[32709]: pgmap v236: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:47.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:47 smithi102 ceph-mon[29216]: pgmap v237: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:20:47.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:47 smithi161 ceph-mon[32709]: pgmap v237: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:20:47.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:47 smithi183 ceph-mon[33137]: pgmap v237: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:20:49.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:49 smithi102 ceph-mon[29216]: pgmap v238: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:49 smithi161 ceph-mon[32709]: pgmap v238: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:49.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:49 smithi183 ceph-mon[33137]: pgmap v238: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:51.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:51 smithi102 ceph-mon[29216]: pgmap v239: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:51.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:51 smithi161 ceph-mon[32709]: pgmap v239: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:51.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:51 smithi183 ceph-mon[33137]: pgmap v239: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:53.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:53 smithi102 ceph-mon[29216]: pgmap v240: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:53.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:53 smithi161 ceph-mon[32709]: pgmap v240: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:53.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:53 smithi183 ceph-mon[33137]: pgmap v240: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:55.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:55 smithi102 ceph-mon[29216]: pgmap v241: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:55.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:55 smithi161 ceph-mon[32709]: pgmap v241: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:55.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:55 smithi183 ceph-mon[33137]: pgmap v241: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:57.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:57 smithi102 ceph-mon[29216]: pgmap v242: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:57.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:57 smithi161 ceph-mon[32709]: pgmap v242: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:57.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:57 smithi183 ceph-mon[33137]: pgmap v242: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.651+0000 7f4a9d390640 1 -- 172.21.15.102:0/483668215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a9810bd10 msgr2=0x7f4a9810e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.651+0000 7f4a9d390640 1 --2- 172.21.15.102:0/483668215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a9810bd10 0x7f4a9810e1b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009ca0 tx=0x7f4a9002a9e0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.651+0000 7f4a9d390640 1 -- 172.21.15.102:0/483668215 shutdown_connections 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.651+0000 7f4a9d390640 1 --2- 172.21.15.102:0/483668215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a9810bd10 0x7f4a9810e1b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.651+0000 7f4a9d390640 1 --2- 172.21.15.102:0/483668215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a981032f0 0x7f4a9810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.651+0000 7f4a9d390640 1 --2- 172.21.15.102:0/483668215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a981029d0 0x7f4a98102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.651+0000 7f4a9d390640 1 -- 172.21.15.102:0/483668215 >> 172.21.15.102:0/483668215 conn(0x7f4a980777c0 msgr2=0x7f4a98077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.651+0000 7f4a9d390640 1 -- 172.21.15.102:0/483668215 shutdown_connections 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.651+0000 7f4a9d390640 1 -- 172.21.15.102:0/483668215 wait complete. 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.652+0000 7f4a9d390640 1 Processor -- start 2024-04-24T23:20:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.652+0000 7f4a9d390640 1 -- start start 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.652+0000 7f4a9d390640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a981029d0 0x7f4a98197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.652+0000 7f4a9d390640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a981032f0 0x7f4a981983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.652+0000 7f4a9d390640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9810bd10 0x7f4a9819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.652+0000 7f4a9d390640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981119c0 con 0x7f4a981032f0 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.652+0000 7f4a9d390640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98111840 con 0x7f4a9810bd10 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.652+0000 7f4a9d390640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98111b40 con 0x7f4a981029d0 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a967fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a981032f0 0x7f4a981983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a967fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a981032f0 0x7f4a981983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43652/0 (socket says 172.21.15.102:43652) 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a967fc640 1 -- 172.21.15.102:0/1926163280 learned_addr learned my addr 172.21.15.102:0/1926163280 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a981029d0 0x7f4a98197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a977fe640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9810bd10 0x7f4a9819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a967fc640 1 -- 172.21.15.102:0/1926163280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a981029d0 msgr2=0x7f4a98197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a967fc640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a981029d0 0x7f4a98197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a967fc640 1 -- 172.21.15.102:0/1926163280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9810bd10 msgr2=0x7f4a9819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a967fc640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9810bd10 0x7f4a9819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a967fc640 1 -- 172.21.15.102:0/1926163280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9819ce70 con 0x7f4a981032f0 2024-04-24T23:20:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a967fc640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a981032f0 0x7f4a981983c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00d950 tx=0x7f4a8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a7ffff640 1 -- 172.21.15.102:0/1926163280 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c017070 con 0x7f4a981032f0 2024-04-24T23:20:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a7ffff640 1 -- 172.21.15.102:0/1926163280 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a8c009c30 con 0x7f4a981032f0 2024-04-24T23:20:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a7ffff640 1 -- 172.21.15.102:0/1926163280 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c00f620 con 0x7f4a981032f0 2024-04-24T23:20:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a9d390640 1 -- 172.21.15.102:0/1926163280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9819d160 con 0x7f4a981032f0 2024-04-24T23:20:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.653+0000 7f4a9d390640 1 -- 172.21.15.102:0/1926163280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a981a49f0 con 0x7f4a981032f0 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4a7ffff640 1 -- 172.21.15.102:0/1926163280 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f4a8c0109a0 con 0x7f4a981032f0 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4aa017c740 1 Processor -- start 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4aa017c740 1 -- start start 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642c240 0x55a70642c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642cb60 0x55a706436cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706439620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4aa017c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a706390cb0 con 0x55a70642cb60 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4aa017c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7063395b0 con 0x55a70642c240 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4aa017c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a70638cc70 con 0x55a706437230 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642cb60 0x55a706436cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642cb60 0x55a706436cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43664/0 (socket says 172.21.15.102:43664) 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4a96ffd640 1 -- 172.21.15.102:0/1184166416 learned_addr learned my addr 172.21.15.102:0/1184166416 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4a7ffff640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f4a800a2e20 0x7f4a800a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.656+0000 7f4a967fc640 1 --2- 172.21.15.102:0/1184166416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706439620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a977fe640 1 --2- 172.21.15.102:0/1184166416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642c240 0x55a70642c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a977fe640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f4a800a2e20 0x7f4a800a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a96ffd640 1 -- 172.21.15.102:0/1184166416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 msgr2=0x55a706439620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/1184166416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706439620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a96ffd640 1 -- 172.21.15.102:0/1184166416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642c240 msgr2=0x55a70642c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/1184166416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642c240 0x55a70642c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a96ffd640 1 -- 172.21.15.102:0/1184166416 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7060a5210 con 0x55a70642cb60 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a7ffff640 1 -- 172.21.15.102:0/1926163280 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7f4a8c0cefc0 con 0x7f4a981032f0 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/1184166416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642cb60 0x55a706436cf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009d20 tx=0x7f4a88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9535b7599e84b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a7d7fa640 1 -- 172.21.15.102:0/1184166416 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8801e070 con 0x55a70642cb60 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a7d7fa640 1 -- 172.21.15.102:0/1184166416 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a88019ae0 con 0x55a70642cb60 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a7d7fa640 1 -- 172.21.15.102:0/1184166416 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8800eac0 con 0x55a70642cb60 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.657+0000 7f4a977fe640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f4a800a2e20 0x7f4a800a52e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005cc0 tx=0x7f4a9003a040 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 -- 172.21.15.102:0/1184166416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642cb60 msgr2=0x55a706436cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 --2- 172.21.15.102:0/1184166416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642cb60 0x55a706436cf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009d20 tx=0x7f4a88011810 comp rx=0 tx=0).stop 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 -- 172.21.15.102:0/1184166416 shutdown_connections 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 --2- 172.21.15.102:0/1184166416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706439620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 --2- 172.21.15.102:0/1184166416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642cb60 0x55a706436cf0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 --2- 172.21.15.102:0/1184166416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642c240 0x55a70642c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 -- 172.21.15.102:0/1184166416 >> 172.21.15.102:0/1184166416 conn(0x55a706428560 msgr2=0x55a706428940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 -- 172.21.15.102:0/1184166416 shutdown_connections 2024-04-24T23:20:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 -- 172.21.15.102:0/1184166416 wait complete. 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 -- start start 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 Processor -- start 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706434160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.658+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a7064346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706434be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4aa017c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a70643e3a0 con 0x55a70642c240 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4aa017c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a70634cb50 con 0x55a70642cb60 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4aa017c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a70643e520 con 0x55a706437230 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4aa017c740 10 client.0 _ll_register_callbacks cb 0x55a706387ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706434160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706434160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43670/0 (socket says 172.21.15.102:43670) 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a96ffd640 1 -- 172.21.15.102:0/3780829258 learned_addr learned my addr 172.21.15.102:0/3780829258 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a977fe640 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706434be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a967fc640 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a7064346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a96ffd640 1 -- 172.21.15.102:0/3780829258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 msgr2=0x55a706434be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706434be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a96ffd640 1 -- 172.21.15.102:0/3780829258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 msgr2=0x55a7064346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a7064346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a96ffd640 1 -- 172.21.15.102:0/3780829258 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a706171620 con 0x55a70642c240 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706434160 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009710 tx=0x7f4a88009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e85dcd3b2cf9d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a7aff5640 1 -- 172.21.15.102:0/3780829258 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88026020 con 0x55a70642c240 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a7aff5640 1 -- 172.21.15.102:0/3780829258 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7f4a88019840 con 0x55a70642c240 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4aa017c740 1 -- 172.21.15.102:0/3780829258 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a88019ae0 con 0x55a70642c240 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4a7aff5640 1 -- 172.21.15.102:0/3780829258 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a88031050 con 0x55a70642c240 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4aa017c740 10 client.16038 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.659+0000 7f4aa017c740 1 -- 172.21.15.102:0/3780829258 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a70644a6d0 con 0x55a70642c240 2024-04-24T23:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4a7aff5640 1 -- 172.21.15.102:0/3780829258 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a88019ae0 con 0x55a70642c240 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4a7aff5640 1 -- 172.21.15.102:0/3780829258 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4a88004b60 con 0x55a70642c240 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4aa017c740 10 client.16038 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4aa017c740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4aa017c740 20 client.16038 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a70644bb30 0x55a70644df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4aa017c740 4 client.16038 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4aa017c740 1 -- 172.21.15.102:0/3780829258 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a70639e1f0 con 0x55a70644bb30 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4a967fc640 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a70644bb30 0x55a70644df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4a967fc640 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a70644bb30 0x55a70644df20 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972d1c6148927f4c server_cookie=ba7c02c8dd487594 in_seq=0 out_seq=0 2024-04-24T23:20:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.660+0000 7f4a7aff5640 10 client.16038 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4a7aff5640 1 -- 172.21.15.102:0/3780829258 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a70639e1f0 con 0x55a70644bb30 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4a7aff5640 10 client.16038 handle_command_reply: tid=0 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 client.16038 shutdown 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 -- 172.21.15.102:0/3780829258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 msgr2=0x55a706434160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706434160 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009710 tx=0x7f4a88009f60 comp rx=0 tx=0).stop 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 -- 172.21.15.102:0/3780829258 shutdown_connections 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706434be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a7064346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a70644bb30 0x55a70644df20 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3780829258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706434160 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4a7aff5640 0 client.16038 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 -- 172.21.15.102:0/3780829258 >> 172.21.15.102:0/3780829258 conn(0x55a706428560 msgr2=0x55a70642e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 -- 172.21.15.102:0/3780829258 shutdown_connections 2024-04-24T23:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.661+0000 7f4aa017c740 1 -- 172.21.15.102:0/3780829258 wait complete. 2024-04-24T23:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.662+0000 7f4aa017c740 20 client.16038 trim_cache size 0 max 16384 2024-04-24T23:20:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4aa017c740 1 Processor -- start 2024-04-24T23:20:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4aa017c740 1 -- start start 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706436890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a7064473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706433300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4aa017c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981ada50 con 0x55a70642c240 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4aa017c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981ab320 con 0x55a70642cb60 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4aa017c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981a9190 con 0x55a706437230 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706436890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4a96ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706436890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43680/0 (socket says 172.21.15.102:43680) 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.678+0000 7f4a96ffd640 1 -- 172.21.15.102:0/3458857063 learned_addr learned my addr 172.21.15.102:0/3458857063 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a977fe640 1 --2- 172.21.15.102:0/3458857063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706433300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a96ffd640 1 -- 172.21.15.102:0/3458857063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 msgr2=0x55a706433300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a967fc640 1 --2- 172.21.15.102:0/3458857063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a7064473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/3458857063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706433300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a96ffd640 1 -- 172.21.15.102:0/3458857063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 msgr2=0x55a7064473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/3458857063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a7064473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a96ffd640 1 -- 172.21.15.102:0/3458857063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70644e290 con 0x55a70642c240 2024-04-24T23:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/3458857063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706436890 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800a010 tx=0x7f4a88009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df44dd10c71c7f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a7cff9640 1 -- 172.21.15.102:0/3458857063 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88015020 con 0x55a70642c240 2024-04-24T23:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a7cff9640 1 -- 172.21.15.102:0/3458857063 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a880054b0 con 0x55a70642c240 2024-04-24T23:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.679+0000 7f4a7cff9640 1 -- 172.21.15.102:0/3458857063 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88026040 con 0x55a70642c240 2024-04-24T23:20:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 -- 172.21.15.102:0/3458857063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 msgr2=0x55a706436890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3458857063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706436890 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800a010 tx=0x7f4a88009620 comp rx=0 tx=0).stop 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 -- 172.21.15.102:0/3458857063 shutdown_connections 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3458857063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a706437230 0x55a706433300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3458857063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a7064473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 --2- 172.21.15.102:0/3458857063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a70642c240 0x55a706436890 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 -- 172.21.15.102:0/3458857063 >> 172.21.15.102:0/3458857063 conn(0x55a706428560 msgr2=0x55a706434120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 -- 172.21.15.102:0/3458857063 shutdown_connections 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 -- 172.21.15.102:0/3458857063 wait complete. 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 -- start start 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.680+0000 7f4aa017c740 1 Processor -- start 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a70642c240 0x55a7064438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a706443df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4aa017c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a706437230 0x55a7064305b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4aa017c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981a97e0 con 0x55a706437230 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4aa017c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981aac50 con 0x55a70642cb60 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4aa017c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981a9eb0 con 0x55a70642c240 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4aa017c740 10 client.0 _ll_register_callbacks cb 0x7f4a8801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a977fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a706437230 0x55a7064305b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a977fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a706437230 0x55a7064305b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43690/0 (socket says 172.21.15.102:43690) 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a977fe640 1 -- 172.21.15.102:0/484137493 learned_addr learned my addr 172.21.15.102:0/484137493 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a967fc640 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a706443df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a70642c240 0x55a7064438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a977fe640 1 -- 172.21.15.102:0/484137493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a70642c240 msgr2=0x55a7064438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a977fe640 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a70642c240 0x55a7064438b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a977fe640 1 -- 172.21.15.102:0/484137493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 msgr2=0x55a706443df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a977fe640 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a706443df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a977fe640 1 -- 172.21.15.102:0/484137493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a706430d60 con 0x55a706437230 2024-04-24T23:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a977fe640 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a706437230 0x55a7064305b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009ca0 tx=0x7f4a9002a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff83497e0acc439 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a7aff5640 1 -- 172.21.15.102:0/484137493 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90057060 con 0x55a706437230 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4a7aff5640 1 -- 172.21.15.102:0/484137493 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8579+0+0 (secure 0 0 0) 0x7f4a9002f840 con 0x55a706437230 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.681+0000 7f4aa017c740 1 -- 172.21.15.102:0/484137493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a880054b0 con 0x55a706437230 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4a7aff5640 1 -- 172.21.15.102:0/484137493 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a90041040 con 0x55a706437230 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4aa017c740 10 client.16044 fetch_fsmap learned FSMap version 26 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4aa017c740 1 -- 172.21.15.102:0/484137493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a7064b1a40 con 0x55a706437230 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4a7aff5640 1 -- 172.21.15.102:0/484137493 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a880054b0 con 0x55a706437230 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4a7aff5640 1 -- 172.21.15.102:0/484137493 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4a90004560 con 0x55a706437230 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4aa017c740 10 client.16044 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4aa017c740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4aa017c740 20 client.16044 populate_metadata read hostname 'smithi102' 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4aa017c740 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a7064b21f0 0x55a7064b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4aa017c740 4 client.16044 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:20:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4aa017c740 1 -- 172.21.15.102:0/484137493 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a706398540 con 0x55a7064b21f0 2024-04-24T23:20:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.682+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a7064b21f0 0x55a7064b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.683+0000 7f4a7aff5640 10 client.16044 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.683+0000 7f4a96ffd640 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a7064b21f0 0x55a7064b4610 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351bd2c416e5f26e server_cookie=d273b6f80c17340c in_seq=0 out_seq=0 2024-04-24T23:20:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4a7aff5640 1 -- 172.21.15.102:0/484137493 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a706398540 con 0x55a7064b21f0 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4a7aff5640 10 client.16044 handle_command_reply: tid=0 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 client.16044 shutdown 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 -- 172.21.15.102:0/484137493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a706437230 msgr2=0x55a7064305b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a706437230 0x55a7064305b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009ca0 tx=0x7f4a9002a9e0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 -- 172.21.15.102:0/484137493 shutdown_connections 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55a7064b21f0 0x55a7064b4610 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a706437230 0x55a7064305b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a70642cb60 0x55a706443df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 --2- 172.21.15.102:0/484137493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a70642c240 0x55a7064438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 -- 172.21.15.102:0/484137493 >> 172.21.15.102:0/484137493 conn(0x55a706428560 msgr2=0x55a70642df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4a7aff5640 0 client.16044 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:20:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 -- 172.21.15.102:0/484137493 shutdown_connections 2024-04-24T23:20:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 1 -- 172.21.15.102:0/484137493 wait complete. 2024-04-24T23:20:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.691+0000 7f4aa017c740 20 client.16044 trim_cache size 0 max 16384 2024-04-24T23:20:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:20:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:20:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:20:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:20:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 -- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f4a800a2e20 msgr2=0x7f4a800a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f4a800a2e20 0x7f4a800a52e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005cc0 tx=0x7f4a9003a040 comp rx=0 tx=0).stop 2024-04-24T23:20:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 -- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a981032f0 msgr2=0x7f4a981983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a981032f0 0x7f4a981983c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00d950 tx=0x7f4a8c00de10 comp rx=0 tx=0).stop 2024-04-24T23:20:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 -- 172.21.15.102:0/1926163280 shutdown_connections 2024-04-24T23:20:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9810bd10 0x7f4a9819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f4a800a2e20 0x7f4a800a52e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a981032f0 0x7f4a981983c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 --2- 172.21.15.102:0/1926163280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a981029d0 0x7f4a98197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 -- 172.21.15.102:0/1926163280 >> 172.21.15.102:0/1926163280 conn(0x7f4a980777c0 msgr2=0x7f4a980fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 -- 172.21.15.102:0/1926163280 shutdown_connections 2024-04-24T23:20:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:20:58.692+0000 7f4a9d390640 1 -- 172.21.15.102:0/1926163280 wait complete. 2024-04-24T23:20:59.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:20:59 smithi183 ceph-mon[33137]: pgmap v243: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:59.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:20:59 smithi102 ceph-mon[29216]: pgmap v243: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:20:59.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:20:59 smithi161 ceph-mon[32709]: pgmap v243: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:21:01.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:01 smithi102 ceph-mon[29216]: pgmap v244: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:01.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:01 smithi161 ceph-mon[32709]: pgmap v244: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:01 smithi183 ceph-mon[33137]: pgmap v244: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:03.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:03 smithi102 ceph-mon[29216]: pgmap v245: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:03.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:03 smithi161 ceph-mon[32709]: pgmap v245: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:03.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:03 smithi183 ceph-mon[33137]: pgmap v245: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:05.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:05 smithi102 ceph-mon[29216]: pgmap v246: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 MiB/s wr, 41 op/s 2024-04-24T23:21:05.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:05 smithi161 ceph-mon[32709]: pgmap v246: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 MiB/s wr, 41 op/s 2024-04-24T23:21:05.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:05 smithi183 ceph-mon[33137]: pgmap v246: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 MiB/s wr, 41 op/s 2024-04-24T23:21:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:07 smithi102 ceph-mon[29216]: pgmap v247: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 24 MiB/s wr, 34 op/s 2024-04-24T23:21:07.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:07 smithi102 ceph-mon[29216]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:21:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:07 smithi161 ceph-mon[32709]: pgmap v247: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 24 MiB/s wr, 34 op/s 2024-04-24T23:21:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:07 smithi161 ceph-mon[32709]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:21:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:07 smithi183 ceph-mon[33137]: pgmap v247: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 24 MiB/s wr, 34 op/s 2024-04-24T23:21:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:07 smithi183 ceph-mon[33137]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:21:09.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:09 smithi102 ceph-mon[29216]: pgmap v249: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 29 MiB/s wr, 40 op/s 2024-04-24T23:21:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:09 smithi161 ceph-mon[32709]: pgmap v249: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 29 MiB/s wr, 40 op/s 2024-04-24T23:21:09.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:09 smithi183 ceph-mon[33137]: pgmap v249: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 29 MiB/s wr, 40 op/s 2024-04-24T23:21:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:10 smithi183 ceph-mon[33137]: pgmap v250: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 29 MiB/s wr, 40 op/s 2024-04-24T23:21:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:10 smithi183 ceph-mon[33137]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:21:11.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:10 smithi102 ceph-mon[29216]: pgmap v250: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 29 MiB/s wr, 40 op/s 2024-04-24T23:21:11.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:10 smithi102 ceph-mon[29216]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:21:11.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:10 smithi161 ceph-mon[32709]: pgmap v250: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 29 MiB/s wr, 40 op/s 2024-04-24T23:21:11.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:10 smithi161 ceph-mon[32709]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:21:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 -- 172.21.15.102:0/739065240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594100620 msgr2=0x7fc594100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 --2- 172.21.15.102:0/739065240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594100620 0x7fc594100aa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc588009930 tx=0x7fc58802a420 comp rx=0 tx=0).stop 2024-04-24T23:21:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 -- 172.21.15.102:0/739065240 shutdown_connections 2024-04-24T23:21:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 --2- 172.21.15.102:0/739065240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594100fe0 0x7fc59410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 --2- 172.21.15.102:0/739065240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594100620 0x7fc594100aa0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 --2- 172.21.15.102:0/739065240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594106630 0x7fc594106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 -- 172.21.15.102:0/739065240 >> 172.21.15.102:0/739065240 conn(0x7fc5940fc110 msgr2=0x7fc5940fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 -- 172.21.15.102:0/739065240 shutdown_connections 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 -- 172.21.15.102:0/739065240 wait complete. 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.813+0000 7fc59a093640 1 Processor -- start 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc59a093640 1 -- start start 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc59a093640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594100620 0x7fc594196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc59a093640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594100fe0 0x7fc594196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc59a093640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594106630 0x7fc594197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc59a093640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941108f0 con 0x7fc594106630 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc59a093640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc594110770 con 0x7fc594100fe0 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc59a093640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc594110a70 con 0x7fc594100620 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594106630 0x7fc594197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594106630 0x7fc594197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43482/0 (socket says 172.21.15.102:43482) 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc593fff640 1 -- 172.21.15.102:0/461440653 learned_addr learned my addr 172.21.15.102:0/461440653 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc592ffd640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594100fe0 0x7fc594196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc5937fe640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594100620 0x7fc594196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc593fff640 1 -- 172.21.15.102:0/461440653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594100620 msgr2=0x7fc594196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc593fff640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594100620 0x7fc594196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc593fff640 1 -- 172.21.15.102:0/461440653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594100fe0 msgr2=0x7fc594196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc593fff640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594100fe0 0x7fc594196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.814+0000 7fc593fff640 1 -- 172.21.15.102:0/461440653 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5941a6310 con 0x7fc594106630 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.815+0000 7fc593fff640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594106630 0x7fc594197520 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00eb10 tx=0x7fc58c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.815+0000 7fc590ff9640 1 -- 172.21.15.102:0/461440653 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c00ce20 con 0x7fc594106630 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.815+0000 7fc590ff9640 1 -- 172.21.15.102:0/461440653 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc58c005020 con 0x7fc594106630 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.815+0000 7fc59a093640 1 -- 172.21.15.102:0/461440653 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5941a6600 con 0x7fc594106630 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.815+0000 7fc590ff9640 1 -- 172.21.15.102:0/461440653 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c010690 con 0x7fc594106630 2024-04-24T23:21:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.815+0000 7fc59a093640 1 -- 172.21.15.102:0/461440653 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5941a6b20 con 0x7fc594106630 2024-04-24T23:21:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.817+0000 7fc590ff9640 1 -- 172.21.15.102:0/461440653 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fc58c0051d0 con 0x7fc594106630 2024-04-24T23:21:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.817+0000 7fc590ff9640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc57c0a2da0 0x7fc57c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.818+0000 7fc5937fe640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc57c0a2da0 0x7fc57c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.818+0000 7fc590ff9640 1 -- 172.21.15.102:0/461440653 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7fc58c0cfba0 con 0x7fc594106630 2024-04-24T23:21:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.818+0000 7fc5937fe640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc57c0a2da0 0x7fc57c0a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc58400ab60 tx=0x7fc58400a2d0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.823+0000 7fc59cd9e740 1 Processor -- start 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.823+0000 7fc59cd9e740 1 -- start start 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427240 0x5581dc427620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 0x5581dc431cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc434620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5581dc38bcb0 con 0x5581dc432230 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5581dc3345b0 con 0x5581dc427240 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581dc387c70 con 0x5581dc427b60 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc434620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc434620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43498/0 (socket says 172.21.15.102:43498) 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc593fff640 1 -- 172.21.15.102:0/2741651932 learned_addr learned my addr 172.21.15.102:0/2741651932 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc5937fe640 1 --2- 172.21.15.102:0/2741651932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427240 0x5581dc427620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc592ffd640 1 --2- 172.21.15.102:0/2741651932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 0x5581dc431cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc593fff640 1 -- 172.21.15.102:0/2741651932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 msgr2=0x5581dc431cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc593fff640 1 --2- 172.21.15.102:0/2741651932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 0x5581dc431cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc593fff640 1 -- 172.21.15.102:0/2741651932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427240 msgr2=0x5581dc427620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc593fff640 1 --2- 172.21.15.102:0/2741651932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427240 0x5581dc427620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc593fff640 1 -- 172.21.15.102:0/2741651932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581dc0a0210 con 0x5581dc432230 2024-04-24T23:21:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc593fff640 1 --2- 172.21.15.102:0/2741651932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc434620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc58c09c010 tx=0x7fc58c09c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0171370d5a1858b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.824+0000 7fc579ffb640 1 -- 172.21.15.102:0/2741651932 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0a4cf0 con 0x5581dc432230 2024-04-24T23:21:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc579ffb640 1 -- 172.21.15.102:0/2741651932 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc58c005580 con 0x5581dc432230 2024-04-24T23:21:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc579ffb640 1 -- 172.21.15.102:0/2741651932 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0986f0 con 0x5581dc432230 2024-04-24T23:21:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2741651932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 msgr2=0x5581dc434620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/2741651932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc434620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc58c09c010 tx=0x7fc58c09c840 comp rx=0 tx=0).stop 2024-04-24T23:21:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2741651932 shutdown_connections 2024-04-24T23:21:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/2741651932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc434620 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/2741651932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 0x5581dc431cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/2741651932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427240 0x5581dc427620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2741651932 >> 172.21.15.102:0/2741651932 conn(0x5581dc423560 msgr2=0x5581dc423940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2741651932 shutdown_connections 2024-04-24T23:21:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.825+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2741651932 wait complete. 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc59cd9e740 1 -- start start 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc59cd9e740 1 Processor -- start 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427240 0x5581dc42f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 0x5581dc42f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc42fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5581dc435020 con 0x5581dc432230 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5581dc347b50 con 0x5581dc427b60 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581dc4351a0 con 0x5581dc427240 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc59cd9e740 10 client.0 _ll_register_callbacks cb 0x5581dc382ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc42fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc592ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 0x5581dc42f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc593fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc42fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43500/0 (socket says 172.21.15.102:43500) 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc593fff640 1 -- 172.21.15.102:0/3013078563 learned_addr learned my addr 172.21.15.102:0/3013078563 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.826+0000 7fc5937fe640 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427240 0x5581dc42f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc592ffd640 1 -- 172.21.15.102:0/3013078563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427240 msgr2=0x5581dc42f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc592ffd640 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427240 0x5581dc42f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc592ffd640 1 -- 172.21.15.102:0/3013078563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 msgr2=0x5581dc42fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc592ffd640 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc42fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc592ffd640 1 -- 172.21.15.102:0/3013078563 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581dc16c620 con 0x5581dc427b60 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc5937fe640 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427240 0x5581dc42f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc593fff640 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc42fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc592ffd640 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 0x5581dc42f720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc588031440 tx=0x7fc5880314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e3b0a8c13a0295 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc572ffd640 1 -- 172.21.15.102:0/3013078563 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588035040 con 0x5581dc427b60 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc59cd9e740 1 -- 172.21.15.102:0/3013078563 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc58c005580 con 0x5581dc427b60 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc572ffd640 1 -- 172.21.15.102:0/3013078563 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7fc588004050 con 0x5581dc427b60 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc572ffd640 1 -- 172.21.15.102:0/3013078563 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc588004250 con 0x5581dc427b60 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc59cd9e740 10 client.25463 fetch_fsmap learned FSMap version 26 2024-04-24T23:21:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc59cd9e740 1 -- 172.21.15.102:0/3013078563 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581dc441240 con 0x5581dc427b60 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.827+0000 7fc572ffd640 1 -- 172.21.15.102:0/3013078563 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc58c005580 con 0x5581dc427b60 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc572ffd640 1 -- 172.21.15.102:0/3013078563 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc58803d5e0 con 0x5581dc427b60 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc59cd9e740 10 client.25463 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc59cd9e740 10 client.25463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc59cd9e740 20 client.25463 populate_metadata read hostname 'smithi102' 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5581dc442700 0x5581dc444b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc59cd9e740 4 client.25463 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc59cd9e740 1 -- 172.21.15.102:0/3013078563 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581dc3991f0 con 0x5581dc442700 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc5937fe640 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5581dc442700 0x5581dc444b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc5937fe640 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5581dc442700 0x5581dc444b10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b086cc1c0182a77 server_cookie=11dfeb42c01fed4 in_seq=0 out_seq=0 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.828+0000 7fc572ffd640 10 client.25463 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc572ffd640 1 -- 172.21.15.102:0/3013078563 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581dc3991f0 con 0x5581dc442700 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc572ffd640 10 client.25463 handle_command_reply: tid=0 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 client.25463 shutdown 2024-04-24T23:21:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 -- 172.21.15.102:0/3013078563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 msgr2=0x5581dc42f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 0x5581dc42f720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc588031440 tx=0x7fc5880314a0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 -- 172.21.15.102:0/3013078563 shutdown_connections 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5581dc442700 0x5581dc444b10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc432230 0x5581dc42fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 0x5581dc42f720 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc572ffd640 0 client.25463 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/3013078563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427240 0x5581dc42f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 -- 172.21.15.102:0/3013078563 >> 172.21.15.102:0/3013078563 conn(0x5581dc423560 msgr2=0x5581dc429360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 -- 172.21.15.102:0/3013078563 shutdown_connections 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 1 -- 172.21.15.102:0/3013078563 wait complete. 2024-04-24T23:21:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.829+0000 7fc59cd9e740 20 client.25463 trim_cache size 0 max 16384 2024-04-24T23:21:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.845+0000 7fc59cd9e740 1 Processor -- start 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.845+0000 7fc59cd9e740 1 -- start start 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.845+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.845+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 0x5581dc42fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.845+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc432230 0x5581dc422440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.845+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941b1e90 con 0x5581dc427240 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.845+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941af760 con 0x5581dc432230 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.845+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941ad5d0 con 0x5581dc427b60 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc5937fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc5937fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43506/0 (socket says 172.21.15.102:43506) 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc5937fe640 1 -- 172.21.15.102:0/2898501941 learned_addr learned my addr 172.21.15.102:0/2898501941 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc592ffd640 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 0x5581dc42fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc593fff640 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc432230 0x5581dc422440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc592ffd640 1 -- 172.21.15.102:0/2898501941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc432230 msgr2=0x5581dc422440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc592ffd640 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc432230 0x5581dc422440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc592ffd640 1 -- 172.21.15.102:0/2898501941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 msgr2=0x5581dc43dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc592ffd640 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc592ffd640 1 -- 172.21.15.102:0/2898501941 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581dc445190 con 0x5581dc427b60 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc5937fe640 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc592ffd640 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 0x5581dc42fa80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc58803d9b0 tx=0x7fc588005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b976b88f5f0f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc5797fa640 1 -- 172.21.15.102:0/2898501941 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588035030 con 0x5581dc427b60 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc5797fa640 1 -- 172.21.15.102:0/2898501941 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc588041070 con 0x5581dc427b60 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.846+0000 7fc5797fa640 1 -- 172.21.15.102:0/2898501941 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58803cd30 con 0x5581dc427b60 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.847+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2898501941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 msgr2=0x5581dc42fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.847+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 0x5581dc42fa80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc58803d9b0 tx=0x7fc588005780 comp rx=0 tx=0).stop 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.847+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2898501941 shutdown_connections 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.847+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc432230 0x5581dc422440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.847+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc427b60 0x5581dc42fa80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.847+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/2898501941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.847+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2898501941 >> 172.21.15.102:0/2898501941 conn(0x5581dc423560 msgr2=0x5581dc421d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.847+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2898501941 shutdown_connections 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.847+0000 7fc59cd9e740 1 -- 172.21.15.102:0/2898501941 wait complete. 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc59cd9e740 1 -- start start 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc59cd9e740 1 Processor -- start 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 0x5581dc42ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc59cd9e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc432230 0x5581dc42bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941adc20 con 0x5581dc427240 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941af090 con 0x5581dc427b60 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc59cd9e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941ae2f0 con 0x5581dc432230 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc59cd9e740 10 client.0 _ll_register_callbacks cb 0x7fc588009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc5937fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc5937fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43510/0 (socket says 172.21.15.102:43510) 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc5937fe640 1 -- 172.21.15.102:0/1600773450 learned_addr learned my addr 172.21.15.102:0/1600773450 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc592ffd640 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 0x5581dc42ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.848+0000 7fc593fff640 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc432230 0x5581dc42bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc5937fe640 1 -- 172.21.15.102:0/1600773450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc432230 msgr2=0x5581dc42bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc5937fe640 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc432230 0x5581dc42bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc5937fe640 1 -- 172.21.15.102:0/1600773450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 msgr2=0x5581dc42ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc5937fe640 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 0x5581dc42ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc5937fe640 1 -- 172.21.15.102:0/1600773450 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581dc42c630 con 0x5581dc427240 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc5937fe640 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43a650 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc58402a750 tx=0x7fc584004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46610dd4025b34c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc572ffd640 1 -- 172.21.15.102:0/1600773450 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584037060 con 0x5581dc427240 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc572ffd640 1 -- 172.21.15.102:0/1600773450 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7fc584010aa0 con 0x5581dc427240 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc59cd9e740 1 -- 172.21.15.102:0/1600773450 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc588041070 con 0x5581dc427240 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc572ffd640 1 -- 172.21.15.102:0/1600773450 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc584016070 con 0x5581dc427240 2024-04-24T23:21:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc59cd9e740 10 client.16062 fetch_fsmap learned FSMap version 26 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc59cd9e740 1 -- 172.21.15.102:0/1600773450 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581dc4a8b90 con 0x5581dc427240 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc572ffd640 1 -- 172.21.15.102:0/1600773450 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc588041070 con 0x5581dc427240 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.849+0000 7fc572ffd640 1 -- 172.21.15.102:0/1600773450 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc584031030 con 0x5581dc427240 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.850+0000 7fc59cd9e740 10 client.16062 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.850+0000 7fc59cd9e740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.850+0000 7fc59cd9e740 20 client.16062 populate_metadata read hostname 'smithi102' 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.850+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5581dc4a92f0 0x5581dc4ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.850+0000 7fc59cd9e740 4 client.16062 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.850+0000 7fc59cd9e740 1 -- 172.21.15.102:0/1600773450 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581dc393540 con 0x5581dc4a92f0 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.850+0000 7fc592ffd640 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5581dc4a92f0 0x5581dc4ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.850+0000 7fc572ffd640 10 client.16062 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.850+0000 7fc592ffd640 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5581dc4a92f0 0x5581dc4ab710 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10116d0d8fd7c089 server_cookie=1d6dc66b2e44bd6a in_seq=0 out_seq=0 2024-04-24T23:21:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc572ffd640 1 -- 172.21.15.102:0/1600773450 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581dc393540 con 0x5581dc4a92f0 2024-04-24T23:21:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc572ffd640 10 client.16062 handle_command_reply: tid=0 2024-04-24T23:21:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 client.16062 shutdown 2024-04-24T23:21:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 -- 172.21.15.102:0/1600773450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 msgr2=0x5581dc43a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43a650 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc58402a750 tx=0x7fc584004730 comp rx=0 tx=0).stop 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 -- 172.21.15.102:0/1600773450 shutdown_connections 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581dc432230 0x5581dc42bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581dc427b60 0x5581dc42ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5581dc4a92f0 0x5581dc4ab710 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 --2- 172.21.15.102:0/1600773450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581dc427240 0x5581dc43a650 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc572ffd640 0 client.16062 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 -- 172.21.15.102:0/1600773450 >> 172.21.15.102:0/1600773450 conn(0x5581dc423560 msgr2=0x5581dc4225c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 -- 172.21.15.102:0/1600773450 shutdown_connections 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 1 -- 172.21.15.102:0/1600773450 wait complete. 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.858+0000 7fc59cd9e740 20 client.16062 trim_cache size 0 max 16384 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:21:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:21:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:21:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:21:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 -- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc57c0a2da0 msgr2=0x7fc57c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc57c0a2da0 0x7fc57c0a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc58400ab60 tx=0x7fc58400a2d0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 -- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594106630 msgr2=0x7fc594197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594106630 0x7fc594197520 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00eb10 tx=0x7fc58c00ef10 comp rx=0 tx=0).stop 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 -- 172.21.15.102:0/461440653 shutdown_connections 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc57c0a2da0 0x7fc57c0a5260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc594106630 0x7fc594197520 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594100fe0 0x7fc594196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 --2- 172.21.15.102:0/461440653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594100620 0x7fc594196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 -- 172.21.15.102:0/461440653 >> 172.21.15.102:0/461440653 conn(0x7fc5940fc110 msgr2=0x7fc5940fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 -- 172.21.15.102:0/461440653 shutdown_connections 2024-04-24T23:21:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:11.859+0000 7fc59a093640 1 -- 172.21.15.102:0/461440653 wait complete. 2024-04-24T23:21:13.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:12 smithi183 ceph-mon[33137]: pgmap v252: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 11 op/s 2024-04-24T23:21:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:12 smithi102 ceph-mon[29216]: pgmap v252: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 11 op/s 2024-04-24T23:21:13.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:12 smithi161 ceph-mon[32709]: pgmap v252: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 11 op/s 2024-04-24T23:21:14.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:13 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:14.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:13 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:14.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:13 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:15.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:14 smithi183 ceph-mon[33137]: pgmap v253: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:21:15.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:14 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:15.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:14 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:15.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:14 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:21:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:14 smithi102 ceph-mon[29216]: pgmap v253: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:21:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:14 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:14 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:14 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:21:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:14 smithi161 ceph-mon[32709]: pgmap v253: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 MiB/s wr, 26 op/s 2024-04-24T23:21:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:14 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:14 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:14 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:21:17.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:16 smithi183 ceph-mon[33137]: pgmap v254: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 22 op/s 2024-04-24T23:21:17.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:16 smithi102 ceph-mon[29216]: pgmap v254: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 22 op/s 2024-04-24T23:21:17.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:16 smithi161 ceph-mon[32709]: pgmap v254: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 22 op/s 2024-04-24T23:21:19.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:18 smithi183 ceph-mon[33137]: pgmap v255: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 20 op/s 2024-04-24T23:21:19.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:18 smithi102 ceph-mon[29216]: pgmap v255: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 20 op/s 2024-04-24T23:21:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:18 smithi161 ceph-mon[32709]: pgmap v255: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 20 op/s 2024-04-24T23:21:21.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:20 smithi183 ceph-mon[33137]: pgmap v256: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 20 op/s 2024-04-24T23:21:21.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:20 smithi102 ceph-mon[29216]: pgmap v256: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 20 op/s 2024-04-24T23:21:21.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:20 smithi161 ceph-mon[32709]: pgmap v256: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 20 op/s 2024-04-24T23:21:23.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:22 smithi183 ceph-mon[33137]: pgmap v257: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 17 op/s 2024-04-24T23:21:23.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:22 smithi102 ceph-mon[29216]: pgmap v257: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 17 op/s 2024-04-24T23:21:23.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:22 smithi161 ceph-mon[32709]: pgmap v257: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 17 op/s 2024-04-24T23:21:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.956+0000 7f34bb6da640 1 -- 172.21.15.102:0/3033139366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 msgr2=0x7f34b4102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.956+0000 7f34bb6da640 1 --2- 172.21.15.102:0/3033139366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 0x7f34b4102db0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009900 tx=0x7f34b002a510 comp rx=0 tx=0).stop 2024-04-24T23:21:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.956+0000 7f34bb6da640 1 -- 172.21.15.102:0/3033139366 shutdown_connections 2024-04-24T23:21:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.956+0000 7f34bb6da640 1 --2- 172.21.15.102:0/3033139366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34b410bd10 0x7f34b410e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.956+0000 7f34bb6da640 1 --2- 172.21.15.102:0/3033139366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b41032f0 0x7f34b410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.956+0000 7f34bb6da640 1 --2- 172.21.15.102:0/3033139366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 0x7f34b4102db0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.956+0000 7f34bb6da640 1 -- 172.21.15.102:0/3033139366 >> 172.21.15.102:0/3033139366 conn(0x7f34b40777c0 msgr2=0x7f34b4077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.956+0000 7f34bb6da640 1 -- 172.21.15.102:0/3033139366 shutdown_connections 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.957+0000 7f34bb6da640 1 -- 172.21.15.102:0/3033139366 wait complete. 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.957+0000 7f34bb6da640 1 Processor -- start 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.957+0000 7f34bb6da640 1 -- start start 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.957+0000 7f34bb6da640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 0x7f34b4197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.957+0000 7f34bb6da640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34b41032f0 0x7f34b4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.957+0000 7f34bb6da640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b410bd10 0x7f34b419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.957+0000 7f34bb6da640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41119c0 con 0x7f34b41029d0 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.957+0000 7f34bb6da640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4111840 con 0x7f34b410bd10 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.957+0000 7f34bb6da640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4111b40 con 0x7f34b41032f0 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9476640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 0x7f34b4197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9476640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 0x7f34b4197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50218/0 (socket says 172.21.15.102:50218) 2024-04-24T23:21:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9476640 1 -- 172.21.15.102:0/1807332858 learned_addr learned my addr 172.21.15.102:0/1807332858 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b8c75640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34b41032f0 0x7f34b4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9c77640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b410bd10 0x7f34b419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9476640 1 -- 172.21.15.102:0/1807332858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34b41032f0 msgr2=0x7f34b4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9476640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34b41032f0 0x7f34b4198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9476640 1 -- 172.21.15.102:0/1807332858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b410bd10 msgr2=0x7f34b419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9476640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b410bd10 0x7f34b419c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9476640 1 -- 172.21.15.102:0/1807332858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b419cfb0 con 0x7f34b41029d0 2024-04-24T23:21:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34b9476640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 0x7f34b4197fc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f34b0006010 tx=0x7f34b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34aa7fc640 1 -- 172.21.15.102:0/1807332858 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b003f030 con 0x7f34b41029d0 2024-04-24T23:21:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34aa7fc640 1 -- 172.21.15.102:0/1807332858 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34b00035f0 con 0x7f34b41029d0 2024-04-24T23:21:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34bb6da640 1 -- 172.21.15.102:0/1807332858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b419d240 con 0x7f34b41029d0 2024-04-24T23:21:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34aa7fc640 1 -- 172.21.15.102:0/1807332858 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0035070 con 0x7f34b41029d0 2024-04-24T23:21:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.958+0000 7f34bb6da640 1 -- 172.21.15.102:0/1807332858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b41a4b30 con 0x7f34b41029d0 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.961+0000 7f34be3ee740 1 Processor -- start 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.961+0000 7f34aa7fc640 1 -- 172.21.15.102:0/1807332858 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f34b0003b10 con 0x7f34b41029d0 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.961+0000 7f34be3ee740 1 -- start start 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069120 0x55eb0f069500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069a40 0x55eb0f078100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 0x55eb0f07aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34be3ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55eb0efcdcb0 con 0x55eb0f069a40 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34be3ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb0ef765b0 con 0x55eb0f078640 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34be3ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb0efc9c70 con 0x55eb0f069120 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9476640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069a40 0x55eb0f078100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9476640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069a40 0x55eb0f078100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50232/0 (socket says 172.21.15.102:50232) 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9476640 1 -- 172.21.15.102:0/4179260552 learned_addr learned my addr 172.21.15.102:0/4179260552 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b8c75640 1 --2- 172.21.15.102:0/4179260552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 0x55eb0f07aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9c77640 1 --2- 172.21.15.102:0/4179260552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069120 0x55eb0f069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34aa7fc640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f34a00a2da0 0x7f34a00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9476640 1 -- 172.21.15.102:0/4179260552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069120 msgr2=0x55eb0f069500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9476640 1 --2- 172.21.15.102:0/4179260552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069120 0x55eb0f069500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9476640 1 -- 172.21.15.102:0/4179260552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 msgr2=0x55eb0f07aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9476640 1 --2- 172.21.15.102:0/4179260552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 0x55eb0f07aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9476640 1 -- 172.21.15.102:0/4179260552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0ece2210 con 0x55eb0f069a40 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.962+0000 7f34b9c77640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f34a00a2da0 0x7f34a00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.963+0000 7f34aa7fc640 1 -- 172.21.15.102:0/1807332858 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7f34b00efde0 con 0x7f34b41029d0 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.963+0000 7f34b9476640 1 --2- 172.21.15.102:0/4179260552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069a40 0x55eb0f078100 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f34b00ed460 tx=0x7f34b0086ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e203383cfac22c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.963+0000 7f349f7fe640 1 -- 172.21.15.102:0/4179260552 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0049070 con 0x55eb0f069a40 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.963+0000 7f349f7fe640 1 -- 172.21.15.102:0/4179260552 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34b00bc9c0 con 0x55eb0f069a40 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.963+0000 7f349f7fe640 1 -- 172.21.15.102:0/4179260552 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b008b000 con 0x55eb0f069a40 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.963+0000 7f34b9c77640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f34a00a2da0 0x7f34a00a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00bd50 tx=0x7f34ac004a40 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 -- 172.21.15.102:0/4179260552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069a40 msgr2=0x55eb0f078100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 --2- 172.21.15.102:0/4179260552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069a40 0x55eb0f078100 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f34b00ed460 tx=0x7f34b0086ef0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 -- 172.21.15.102:0/4179260552 shutdown_connections 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 --2- 172.21.15.102:0/4179260552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 0x55eb0f07aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 --2- 172.21.15.102:0/4179260552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069a40 0x55eb0f078100 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 --2- 172.21.15.102:0/4179260552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069120 0x55eb0f069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 -- 172.21.15.102:0/4179260552 >> 172.21.15.102:0/4179260552 conn(0x55eb0f065560 msgr2=0x55eb0f065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 -- 172.21.15.102:0/4179260552 shutdown_connections 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 -- 172.21.15.102:0/4179260552 wait complete. 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 -- start start 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 Processor -- start 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.964+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 0x55eb0f078100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f075490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34be3ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55eb0f07b240 con 0x55eb0f078640 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34be3ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb0ef89b50 con 0x55eb0f069120 2024-04-24T23:21:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34be3ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb0f07b3c0 con 0x55eb0f069a40 2024-04-24T23:21:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34be3ee740 10 client.0 _ll_register_callbacks cb 0x55eb0efc4ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9c77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9c77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f075bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50242/0 (socket says 172.21.15.102:50242) 2024-04-24T23:21:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9c77640 1 -- 172.21.15.102:0/1834504617 learned_addr learned my addr 172.21.15.102:0/1834504617 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9476640 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 0x55eb0f078100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b8c75640 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9c77640 1 -- 172.21.15.102:0/1834504617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 msgr2=0x55eb0f075490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9c77640 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f075490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9c77640 1 -- 172.21.15.102:0/1834504617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 msgr2=0x55eb0f078100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9c77640 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 0x55eb0f078100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9c77640 1 -- 172.21.15.102:0/1834504617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0edae620 con 0x55eb0f078640 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34b9c77640 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f075bd0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f34ac011e40 tx=0x7f34ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476b8c88ae049b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f349cff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f349cff9640 1 -- 172.21.15.102:0/1834504617 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac01c430 con 0x55eb0f078640 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f349cff9640 1 -- 172.21.15.102:0/1834504617 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7f34ac01c5d0 con 0x55eb0f078640 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.965+0000 7f34be3ee740 1 -- 172.21.15.102:0/1834504617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b00bc9c0 con 0x55eb0f078640 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f349cff9640 1 -- 172.21.15.102:0/1834504617 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34ac01c7d0 con 0x55eb0f078640 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f34be3ee740 10 client.16074 fetch_fsmap learned FSMap version 26 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f34be3ee740 1 -- 172.21.15.102:0/1834504617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb0f0872d0 con 0x55eb0f078640 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f349cff9640 1 -- 172.21.15.102:0/1834504617 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34b00bc9c0 con 0x55eb0f078640 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f349cff9640 1 -- 172.21.15.102:0/1834504617 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f34ac01da10 con 0x55eb0f078640 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f34be3ee740 10 client.16074 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:21:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f34be3ee740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f34be3ee740 20 client.16074 populate_metadata read hostname 'smithi102' 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb0f088850 0x55eb0f08ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f34be3ee740 4 client.16074 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f34be3ee740 1 -- 172.21.15.102:0/1834504617 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb0efdb1f0 con 0x55eb0f088850 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.966+0000 7f34b9476640 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb0f088850 0x55eb0f08ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.967+0000 7f34b9476640 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb0f088850 0x55eb0f08ac40 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90bf6c33cbe7c14 server_cookie=24efb67757aa4c95 in_seq=0 out_seq=0 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.967+0000 7f349cff9640 10 client.16074 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.967+0000 7f349cff9640 1 -- 172.21.15.102:0/1834504617 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb0efdb1f0 con 0x55eb0f088850 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.967+0000 7f349cff9640 10 client.16074 handle_command_reply: tid=0 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 client.16074 shutdown 2024-04-24T23:21:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 -- 172.21.15.102:0/1834504617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 msgr2=0x55eb0f075bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f075bd0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f34ac011e40 tx=0x7f34ac004ab0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 -- 172.21.15.102:0/1834504617 shutdown_connections 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb0f088850 0x55eb0f08ac40 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f075bd0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f075490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1834504617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 0x55eb0f078100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 -- 172.21.15.102:0/1834504617 >> 172.21.15.102:0/1834504617 conn(0x55eb0f065560 msgr2=0x55eb0f06b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f349cff9640 0 client.16074 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 -- 172.21.15.102:0/1834504617 shutdown_connections 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 1 -- 172.21.15.102:0/1834504617 wait complete. 2024-04-24T23:21:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.968+0000 7f34be3ee740 20 client.16074 trim_cache size 0 max 16384 2024-04-24T23:21:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.990+0000 7f34be3ee740 1 Processor -- start 2024-04-24T23:21:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34be3ee740 1 -- start start 2024-04-24T23:21:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 0x55eb0f090530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f090a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f073d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34be3ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41adc10 con 0x55eb0f078640 2024-04-24T23:21:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34be3ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41ab4e0 con 0x55eb0f069120 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34be3ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41a9350 con 0x55eb0f069a40 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b9c77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b8c75640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f090a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b9c77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f073d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50246/0 (socket says 172.21.15.102:50246) 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b9c77640 1 -- 172.21.15.102:0/1900313632 learned_addr learned my addr 172.21.15.102:0/1900313632 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b9476640 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 0x55eb0f090530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b8c75640 1 -- 172.21.15.102:0/1900313632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 msgr2=0x55eb0f090530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b8c75640 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 0x55eb0f090530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b8c75640 1 -- 172.21.15.102:0/1900313632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 msgr2=0x55eb0f073d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b8c75640 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f073d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b8c75640 1 -- 172.21.15.102:0/1900313632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0f069870 con 0x55eb0f069a40 2024-04-24T23:21:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b9476640 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 0x55eb0f090530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b9c77640 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f073d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.991+0000 7f34b8c75640 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f090a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55eb0f074cf0 tx=0x7f34a400a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf2a7304fb239fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f349effd640 1 -- 172.21.15.102:0/1900313632 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a401e030 con 0x55eb0f069a40 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f349effd640 1 -- 172.21.15.102:0/1900313632 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34a40054d0 con 0x55eb0f069a40 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f349effd640 1 -- 172.21.15.102:0/1900313632 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a401a660 con 0x55eb0f069a40 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f34be3ee740 1 -- 172.21.15.102:0/1900313632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 msgr2=0x55eb0f090a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f090a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55eb0f074cf0 tx=0x7f34a400a2d0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f34be3ee740 1 -- 172.21.15.102:0/1900313632 shutdown_connections 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f078640 0x55eb0f073d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f090a70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1900313632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f069120 0x55eb0f090530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f34be3ee740 1 -- 172.21.15.102:0/1900313632 >> 172.21.15.102:0/1900313632 conn(0x55eb0f065560 msgr2=0x55eb0f070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f34be3ee740 1 -- 172.21.15.102:0/1900313632 shutdown_connections 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.992+0000 7f34be3ee740 1 -- 172.21.15.102:0/1900313632 wait complete. 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34be3ee740 1 -- start start 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34be3ee740 1 Processor -- start 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069120 0x55eb0f072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34be3ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 0x55eb0f072df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34be3ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41a99a0 con 0x55eb0f069120 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34be3ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41aae10 con 0x55eb0f078640 2024-04-24T23:21:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34be3ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41aa070 con 0x55eb0f069a40 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34be3ee740 10 client.0 _ll_register_callbacks cb 0x7f34ac00b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34b9476640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069120 0x55eb0f072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34b9476640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069120 0x55eb0f072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50260/0 (socket says 172.21.15.102:50260) 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34b9476640 1 -- 172.21.15.102:0/1814396001 learned_addr learned my addr 172.21.15.102:0/1814396001 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34b8c75640 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.993+0000 7f34b9c77640 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 0x55eb0f072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34b9476640 1 -- 172.21.15.102:0/1814396001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 msgr2=0x55eb0f0728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34b9476640 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f0728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34b9476640 1 -- 172.21.15.102:0/1814396001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 msgr2=0x55eb0f072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34b9476640 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 0x55eb0f072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34b9476640 1 -- 172.21.15.102:0/1814396001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0f0edfa0 con 0x55eb0f069120 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34b9c77640 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 0x55eb0f072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34b9476640 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069120 0x55eb0f072370 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f34b0031440 tx=0x7f34b00f3b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6347e6747deeca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f349cff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:21:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f349cff9640 1 -- 172.21.15.102:0/1814396001 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0049040 con 0x55eb0f069120 2024-04-24T23:21:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f349cff9640 1 -- 172.21.15.102:0/1814396001 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7f34b00d0420 con 0x55eb0f069120 2024-04-24T23:21:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34be3ee740 1 -- 172.21.15.102:0/1814396001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a40054d0 con 0x55eb0f069120 2024-04-24T23:21:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f349cff9640 1 -- 172.21.15.102:0/1814396001 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34b00037a0 con 0x55eb0f069120 2024-04-24T23:21:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34be3ee740 10 client.16080 fetch_fsmap learned FSMap version 26 2024-04-24T23:21:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34be3ee740 1 -- 172.21.15.102:0/1814396001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb0f0ef290 con 0x55eb0f069120 2024-04-24T23:21:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f349cff9640 1 -- 172.21.15.102:0/1814396001 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34a40054d0 con 0x55eb0f069120 2024-04-24T23:21:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f349cff9640 1 -- 172.21.15.102:0/1814396001 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f34b00ed770 con 0x55eb0f069120 2024-04-24T23:21:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34be3ee740 10 client.16080 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:21:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34be3ee740 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:21:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34be3ee740 20 client.16080 populate_metadata read hostname 'smithi102' 2024-04-24T23:21:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb0f0ef690 0x55eb0f0f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34be3ee740 4 client.16080 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:21:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.994+0000 7f34be3ee740 1 -- 172.21.15.102:0/1814396001 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb0efd5540 con 0x55eb0f0ef690 2024-04-24T23:21:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.995+0000 7f34b8c75640 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb0f0ef690 0x55eb0f0f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.995+0000 7f349cff9640 10 client.16080 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:24.995+0000 7f34b8c75640 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb0f0ef690 0x55eb0f0f1ab0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc26b5b6b23100ae server_cookie=a97b152479688516 in_seq=0 out_seq=0 2024-04-24T23:21:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f349cff9640 1 -- 172.21.15.102:0/1814396001 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb0efd5540 con 0x55eb0f0ef690 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f349cff9640 10 client.16080 handle_command_reply: tid=0 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 client.16080 shutdown 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 -- 172.21.15.102:0/1814396001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069120 msgr2=0x55eb0f072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069120 0x55eb0f072370 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f34b0031440 tx=0x7f34b00f3b40 comp rx=0 tx=0).stop 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 -- 172.21.15.102:0/1814396001 shutdown_connections 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb0f078640 0x55eb0f072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb0f069a40 0x55eb0f0728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb0f0ef690 0x55eb0f0f1ab0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 --2- 172.21.15.102:0/1814396001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb0f069120 0x55eb0f072370 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f349cff9640 0 client.16080 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 -- 172.21.15.102:0/1814396001 >> 172.21.15.102:0/1814396001 conn(0x55eb0f065560 msgr2=0x55eb0f063d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.003+0000 7f34be3ee740 1 -- 172.21.15.102:0/1814396001 shutdown_connections 2024-04-24T23:21:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34be3ee740 1 -- 172.21.15.102:0/1814396001 wait complete. 2024-04-24T23:21:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34be3ee740 20 client.16080 trim_cache size 0 max 16384 2024-04-24T23:21:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:21:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:21:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:21:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:21:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 -- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f34a00a2da0 msgr2=0x7f34a00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f34a00a2da0 0x7f34a00a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00bd50 tx=0x7f34ac004a40 comp rx=0 tx=0).stop 2024-04-24T23:21:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 -- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 msgr2=0x7f34b4197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 0x7f34b4197fc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f34b0006010 tx=0x7f34b0004270 comp rx=0 tx=0).stop 2024-04-24T23:21:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 -- 172.21.15.102:0/1807332858 shutdown_connections 2024-04-24T23:21:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34b410bd10 0x7f34b419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34b41032f0 0x7f34b4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f34a00a2da0 0x7f34a00a5260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 --2- 172.21.15.102:0/1807332858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34b41029d0 0x7f34b4197fc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 -- 172.21.15.102:0/1807332858 >> 172.21.15.102:0/1807332858 conn(0x7f34b40777c0 msgr2=0x7f34b40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 -- 172.21.15.102:0/1807332858 shutdown_connections 2024-04-24T23:21:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:25.004+0000 7f34bb6da640 1 -- 172.21.15.102:0/1807332858 wait complete. 2024-04-24T23:21:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:24 smithi183 ceph-mon[33137]: pgmap v258: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:21:25.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:24 smithi102 ceph-mon[29216]: pgmap v258: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:21:25.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:24 smithi161 ceph-mon[32709]: pgmap v258: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 MiB/s wr, 24 op/s 2024-04-24T23:21:27.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:26 smithi183 ceph-mon[33137]: pgmap v259: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:27.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:26 smithi102 ceph-mon[29216]: pgmap v259: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:27.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:26 smithi161 ceph-mon[32709]: pgmap v259: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:29.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:28 smithi183 ceph-mon[33137]: pgmap v260: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:21:29.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:28 smithi102 ceph-mon[29216]: pgmap v260: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:21:29.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:28 smithi161 ceph-mon[32709]: pgmap v260: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:21:31.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:30 smithi183 ceph-mon[33137]: pgmap v261: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:31.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:30 smithi102 ceph-mon[29216]: pgmap v261: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:31.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:30 smithi161 ceph-mon[32709]: pgmap v261: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:33.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:32 smithi183 ceph-mon[33137]: pgmap v262: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:33.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:32 smithi102 ceph-mon[29216]: pgmap v262: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:33.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:32 smithi161 ceph-mon[32709]: pgmap v262: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:35.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:35 smithi183 ceph-mon[33137]: pgmap v263: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:21:35.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:35 smithi102 ceph-mon[29216]: pgmap v263: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:21:35.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:35 smithi161 ceph-mon[32709]: pgmap v263: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:21:37.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:37 smithi183 ceph-mon[33137]: pgmap v264: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:37.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:37 smithi102 ceph-mon[29216]: pgmap v264: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:37.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:37 smithi161 ceph-mon[32709]: pgmap v264: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.102+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1861883914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 msgr2=0x7f9ecc107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.102+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1861883914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 0x7f9ecc107b00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009930 tx=0x7f9ebc02a420 comp rx=0 tx=0).stop 2024-04-24T23:21:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.102+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1861883914 shutdown_connections 2024-04-24T23:21:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.102+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1861883914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ecc100600 0x7f9ecc10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.102+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1861883914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ecc0ffc40 0x7f9ecc1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.102+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1861883914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 0x7f9ecc107b00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.102+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1861883914 >> 172.21.15.102:0/1861883914 conn(0x7f9ecc076bd0 msgr2=0x7f9ecc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.102+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1861883914 shutdown_connections 2024-04-24T23:21:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.102+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1861883914 wait complete. 2024-04-24T23:21:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed29f9640 1 Processor -- start 2024-04-24T23:21:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed29f9640 1 -- start start 2024-04-24T23:21:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed29f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ecc0ffc40 0x7f9ecc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed29f9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ecc100600 0x7f9ecc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed29f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 0x7f9ecc19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed29f9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1119a0 con 0x7f9ecc107720 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed29f9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc111820 con 0x7f9ecc100600 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed29f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc111b20 con 0x7f9ecc0ffc40 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed0f96640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 0x7f9ecc19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed0f96640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 0x7f9ecc19c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57232/0 (socket says 172.21.15.102:57232) 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ed0f96640 1 -- 172.21.15.102:0/1967663001 learned_addr learned my addr 172.21.15.102:0/1967663001 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ecc0ffc40 0x7f9ecc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.103+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ecc100600 0x7f9ecc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ed0f96640 1 -- 172.21.15.102:0/1967663001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ecc0ffc40 msgr2=0x7f9ecc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ecc0ffc40 0x7f9ecc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ed0f96640 1 -- 172.21.15.102:0/1967663001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ecc100600 msgr2=0x7f9ecc1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ecc100600 0x7f9ecc1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ed0f96640 1 -- 172.21.15.102:0/1967663001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc19ce90 con 0x7f9ecc107720 2024-04-24T23:21:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 0x7f9ecc19c760 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400eaa0 tx=0x7f9ec400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ec97fa640 1 -- 172.21.15.102:0/1967663001 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400cdb0 con 0x7f9ecc107720 2024-04-24T23:21:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1967663001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc19d180 con 0x7f9ecc107720 2024-04-24T23:21:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ec97fa640 1 -- 172.21.15.102:0/1967663001 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ec4005020 con 0x7f9ecc107720 2024-04-24T23:21:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ec97fa640 1 -- 172.21.15.102:0/1967663001 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec40116d0 con 0x7f9ecc107720 2024-04-24T23:21:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.104+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1967663001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc100d50 con 0x7f9ecc107720 2024-04-24T23:21:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.107+0000 7f9ec97fa640 1 -- 172.21.15.102:0/1967663001 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f9ec40051d0 con 0x7f9ecc107720 2024-04-24T23:21:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.107+0000 7f9ec97fa640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9eb40a2da0 0x7f9eb40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.107+0000 7f9ed57e1740 1 Processor -- start 2024-04-24T23:21:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9eb40a2da0 0x7f9eb40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ed57e1740 1 -- start start 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ec97fa640 1 -- 172.21.15.102:0/1967663001 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7f9ec40cf560 con 0x7f9ecc107720 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 0x55bca5e07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07b60 0x55bca5e11cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e12230 0x55bca5e14620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bca5d6bcb0 con 0x55bca5e07b60 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bca5d145b0 con 0x55bca5e07240 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bca5d67c70 con 0x55bca5e12230 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07b60 0x55bca5e11cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07b60 0x55bca5e11cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57248/0 (socket says 172.21.15.102:57248) 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecb7fe640 1 -- 172.21.15.102:0/3326382840 learned_addr learned my addr 172.21.15.102:0/3326382840 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9eb40a2da0 0x7f9eb40a5260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009900 tx=0x7f9ebc03c040 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e12230 0x55bca5e14620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 0x55bca5e07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecb7fe640 1 -- 172.21.15.102:0/3326382840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e12230 msgr2=0x55bca5e14620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e12230 0x55bca5e14620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecb7fe640 1 -- 172.21.15.102:0/3326382840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 msgr2=0x55bca5e07620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 0x55bca5e07620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecb7fe640 1 -- 172.21.15.102:0/3326382840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca5a80210 con 0x55bca5e07b60 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 0x55bca5e07620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.108+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e12230 0x55bca5e14620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.109+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07b60 0x55bca5e11cf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009c90 tx=0x7f9ec0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269500d1667e1bef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.109+0000 7f9eb27fc640 1 -- 172.21.15.102:0/3326382840 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec001e070 con 0x55bca5e07b60 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.109+0000 7f9eb27fc640 1 -- 172.21.15.102:0/3326382840 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ec0019ae0 con 0x55bca5e07b60 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.109+0000 7f9eb27fc640 1 -- 172.21.15.102:0/3326382840 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000e9e0 con 0x55bca5e07b60 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 -- 172.21.15.102:0/3326382840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07b60 msgr2=0x55bca5e11cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07b60 0x55bca5e11cf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009c90 tx=0x7f9ec0011810 comp rx=0 tx=0).stop 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 -- 172.21.15.102:0/3326382840 shutdown_connections 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e12230 0x55bca5e14620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07b60 0x55bca5e11cf0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/3326382840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 0x55bca5e07620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 -- 172.21.15.102:0/3326382840 >> 172.21.15.102:0/3326382840 conn(0x55bca5e03560 msgr2=0x55bca5e03940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 -- 172.21.15.102:0/3326382840 shutdown_connections 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 -- 172.21.15.102:0/3326382840 wait complete. 2024-04-24T23:21:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 -- start start 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.110+0000 7f9ed57e1740 1 Processor -- start 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07240 0x55bca5e0f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 0x55bca5e0f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e12230 0x55bca5e0fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bca5e15020 con 0x55bca5e07240 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bca5d27b50 con 0x55bca5e12230 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bca5e151a0 con 0x55bca5e07b60 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ed57e1740 10 client.0 _ll_register_callbacks cb 0x55bca5d62ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecbfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07240 0x55bca5e0f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecbfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07240 0x55bca5e0f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57256/0 (socket says 172.21.15.102:57256) 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecbfff640 1 -- 172.21.15.102:0/1100961276 learned_addr learned my addr 172.21.15.102:0/1100961276 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 0x55bca5e0f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e12230 0x55bca5e0fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecbfff640 1 -- 172.21.15.102:0/1100961276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 msgr2=0x55bca5e0f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 0x55bca5e0f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecbfff640 1 -- 172.21.15.102:0/1100961276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e12230 msgr2=0x55bca5e0fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e12230 0x55bca5e0fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecbfff640 1 -- 172.21.15.102:0/1100961276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca5b4c620 con 0x55bca5e07240 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e12230 0x55bca5e0fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07240 0x55bca5e0f170 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009930 tx=0x7f9ebc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154c4871b11f9e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.111+0000 7f9eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:21:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9eab7fe640 1 -- 172.21.15.102:0/1100961276 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc057060 con 0x55bca5e07240 2024-04-24T23:21:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9eab7fe640 1 -- 172.21.15.102:0/1100961276 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7f9ebc03a670 con 0x55bca5e07240 2024-04-24T23:21:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9ed57e1740 1 -- 172.21.15.102:0/1100961276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec0019ae0 con 0x55bca5e07240 2024-04-24T23:21:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9eab7fe640 1 -- 172.21.15.102:0/1100961276 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ebc035050 con 0x55bca5e07240 2024-04-24T23:21:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9ed57e1740 10 client.16092 fetch_fsmap learned FSMap version 26 2024-04-24T23:21:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9ed57e1740 1 -- 172.21.15.102:0/1100961276 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bca5e211e0 con 0x55bca5e07240 2024-04-24T23:21:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9eab7fe640 1 -- 172.21.15.102:0/1100961276 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ec0019ae0 con 0x55bca5e07240 2024-04-24T23:21:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9eab7fe640 1 -- 172.21.15.102:0/1100961276 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9ebc051070 con 0x55bca5e07240 2024-04-24T23:21:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9ed57e1740 10 client.16092 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:21:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9ed57e1740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:21:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9ed57e1740 20 client.16092 populate_metadata read hostname 'smithi102' 2024-04-24T23:21:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bca5e226f0 0x55bca5e24ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9ed57e1740 4 client.16092 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.112+0000 7f9ed57e1740 1 -- 172.21.15.102:0/1100961276 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bca5d791f0 con 0x55bca5e226f0 2024-04-24T23:21:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.113+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bca5e226f0 0x55bca5e24ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.113+0000 7f9eab7fe640 10 client.16092 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.113+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bca5e226f0 0x55bca5e24ae0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7d8e8f683f00ce server_cookie=35ac6965bdc663d7 in_seq=0 out_seq=0 2024-04-24T23:21:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9eab7fe640 1 -- 172.21.15.102:0/1100961276 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bca5d791f0 con 0x55bca5e226f0 2024-04-24T23:21:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9eab7fe640 10 client.16092 handle_command_reply: tid=0 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 client.16092 shutdown 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 -- 172.21.15.102:0/1100961276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07240 msgr2=0x55bca5e0f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07240 0x55bca5e0f170 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009930 tx=0x7f9ebc000bc0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 -- 172.21.15.102:0/1100961276 shutdown_connections 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e12230 0x55bca5e0fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 0x55bca5e0f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bca5e226f0 0x55bca5e24ae0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/1100961276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e07240 0x55bca5e0f170 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 -- 172.21.15.102:0/1100961276 >> 172.21.15.102:0/1100961276 conn(0x55bca5e03560 msgr2=0x55bca5e09260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9eab7fe640 0 client.16092 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 -- 172.21.15.102:0/1100961276 shutdown_connections 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 1 -- 172.21.15.102:0/1100961276 wait complete. 2024-04-24T23:21:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.114+0000 7f9ed57e1740 20 client.16092 trim_cache size 0 max 16384 2024-04-24T23:21:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed57e1740 1 Processor -- start 2024-04-24T23:21:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed57e1740 1 -- start start 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 0x55bca5e1dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 0x55bca5e099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e0e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1ada80 con 0x55bca5e12230 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1ab350 con 0x55bca5e07240 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1a91c0 con 0x55bca5e07b60 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed0f96640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e0e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed0f96640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e0e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57258/0 (socket says 172.21.15.102:57258) 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ed0f96640 1 -- 172.21.15.102:0/258088207 learned_addr learned my addr 172.21.15.102:0/258088207 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.130+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 0x55bca5e099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 0x55bca5e1dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9ecb7fe640 1 -- 172.21.15.102:0/258088207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 msgr2=0x55bca5e1dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 0x55bca5e1dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9ecb7fe640 1 -- 172.21.15.102:0/258088207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 msgr2=0x55bca5e0e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e0e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9ecb7fe640 1 -- 172.21.15.102:0/258088207 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca5e04130 con 0x55bca5e07b60 2024-04-24T23:21:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e0e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 0x55bca5e099a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0000c00 tx=0x7f9ec000e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aaae2d7e77ad9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9eb1ffb640 1 -- 172.21.15.102:0/258088207 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0031070 con 0x55bca5e07b60 2024-04-24T23:21:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9eb1ffb640 1 -- 172.21.15.102:0/258088207 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ec001f040 con 0x55bca5e07b60 2024-04-24T23:21:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.131+0000 7f9eb1ffb640 1 -- 172.21.15.102:0/258088207 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec001e040 con 0x55bca5e07b60 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 -- 172.21.15.102:0/258088207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 msgr2=0x55bca5e099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 0x55bca5e099a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0000c00 tx=0x7f9ec000e5a0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 -- 172.21.15.102:0/258088207 shutdown_connections 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e0e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07b60 0x55bca5e099a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/258088207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07240 0x55bca5e1dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 -- 172.21.15.102:0/258088207 >> 172.21.15.102:0/258088207 conn(0x55bca5e03560 msgr2=0x55bca5e0f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 -- 172.21.15.102:0/258088207 shutdown_connections 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 -- 172.21.15.102:0/258088207 wait complete. 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.132+0000 7f9ed57e1740 1 -- start start 2024-04-24T23:21:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed57e1740 1 Processor -- start 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07240 0x55bca5e1a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07b60 0x55bca5e1a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed57e1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e15890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1a9810 con 0x55bca5e12230 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1aac80 con 0x55bca5e07b60 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed57e1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1a9ee0 con 0x55bca5e07240 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed57e1740 10 client.0 _ll_register_callbacks cb 0x7f9ebc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed0f96640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e15890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed0f96640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e15890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57274/0 (socket says 172.21.15.102:57274) 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed0f96640 1 -- 172.21.15.102:0/4256536456 learned_addr learned my addr 172.21.15.102:0/4256536456 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07240 0x55bca5e1a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07b60 0x55bca5e1a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed0f96640 1 -- 172.21.15.102:0/4256536456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07240 msgr2=0x55bca5e1a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07240 0x55bca5e1a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed0f96640 1 -- 172.21.15.102:0/4256536456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07b60 msgr2=0x55bca5e1a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07b60 0x55bca5e1a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed0f96640 1 -- 172.21.15.102:0/4256536456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca5e16090 con 0x55bca5e12230 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ecb7fe640 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07b60 0x55bca5e1a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9ed0f96640 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e15890 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40092f0 tx=0x7f9ec400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6aa4b16d1a9f6be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.133+0000 7f9eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9eab7fe640 1 -- 172.21.15.102:0/4256536456 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec401f040 con 0x55bca5e12230 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9ed57e1740 1 -- 172.21.15.102:0/4256536456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec001f040 con 0x55bca5e12230 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9eab7fe640 1 -- 172.21.15.102:0/4256536456 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7f9ec409b140 con 0x55bca5e12230 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9eab7fe640 1 -- 172.21.15.102:0/4256536456 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ec40cf040 con 0x55bca5e12230 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9ed57e1740 10 client.16098 fetch_fsmap learned FSMap version 26 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9ed57e1740 1 -- 172.21.15.102:0/4256536456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bca5e887d0 con 0x55bca5e12230 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9eab7fe640 1 -- 172.21.15.102:0/4256536456 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ec001f040 con 0x55bca5e12230 2024-04-24T23:21:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9eab7fe640 1 -- 172.21.15.102:0/4256536456 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9ec400f040 con 0x55bca5e12230 2024-04-24T23:21:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9ed57e1740 10 client.16098 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:21:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9ed57e1740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:21:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9ed57e1740 20 client.16098 populate_metadata read hostname 'smithi102' 2024-04-24T23:21:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bca5e88f20 0x55bca5e8b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9ed57e1740 4 client.16098 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:21:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.134+0000 7f9ed57e1740 1 -- 172.21.15.102:0/4256536456 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bca5d73540 con 0x55bca5e88f20 2024-04-24T23:21:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.135+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bca5e88f20 0x55bca5e8b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.135+0000 7f9eab7fe640 10 client.16098 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.135+0000 7f9ecbfff640 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bca5e88f20 0x55bca5e8b340 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6b32ce25fec93c server_cookie=f725e04e52bed1d7 in_seq=0 out_seq=0 2024-04-24T23:21:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.142+0000 7f9eab7fe640 1 -- 172.21.15.102:0/4256536456 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bca5d73540 con 0x55bca5e88f20 2024-04-24T23:21:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9eab7fe640 10 client.16098 handle_command_reply: tid=0 2024-04-24T23:21:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 client.16098 shutdown 2024-04-24T23:21:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 -- 172.21.15.102:0/4256536456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 msgr2=0x55bca5e15890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e15890 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40092f0 tx=0x7f9ec400c800 comp rx=0 tx=0).stop 2024-04-24T23:21:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 -- 172.21.15.102:0/4256536456 shutdown_connections 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bca5e88f20 0x55bca5e8b340 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bca5e12230 0x55bca5e15890 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bca5e07b60 0x55bca5e1a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 --2- 172.21.15.102:0/4256536456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bca5e07240 0x55bca5e1a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 -- 172.21.15.102:0/4256536456 >> 172.21.15.102:0/4256536456 conn(0x55bca5e03560 msgr2=0x55bca5e01bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9eab7fe640 0 client.16098 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 -- 172.21.15.102:0/4256536456 shutdown_connections 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 1 -- 172.21.15.102:0/4256536456 wait complete. 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed57e1740 20 client.16098 trim_cache size 0 max 16384 2024-04-24T23:21:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:21:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:21:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:21:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9eb40a2da0 msgr2=0x7f9eb40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9eb40a2da0 0x7f9eb40a5260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009900 tx=0x7f9ebc03c040 comp rx=0 tx=0).stop 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 msgr2=0x7f9ecc19c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.143+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 0x7f9ecc19c760 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400eaa0 tx=0x7f9ec400ef60 comp rx=0 tx=0).stop 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.144+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1967663001 shutdown_connections 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.144+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9eb40a2da0 0x7f9eb40a5260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.144+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ecc107720 0x7f9ecc19c760 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.144+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ecc100600 0x7f9ecc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.144+0000 7f9ed29f9640 1 --2- 172.21.15.102:0/1967663001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ecc0ffc40 0x7f9ecc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.144+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1967663001 >> 172.21.15.102:0/1967663001 conn(0x7f9ecc076bd0 msgr2=0x7f9ecc0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.144+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1967663001 shutdown_connections 2024-04-24T23:21:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:38.144+0000 7f9ed29f9640 1 -- 172.21.15.102:0/1967663001 wait complete. 2024-04-24T23:21:39.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:39 smithi183 ceph-mon[33137]: pgmap v265: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:21:39.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:39 smithi102 ceph-mon[29216]: pgmap v265: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:21:39.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:39 smithi161 ceph-mon[32709]: pgmap v265: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 24 op/s 2024-04-24T23:21:41.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:41 smithi183 ceph-mon[33137]: pgmap v266: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:41.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:41 smithi102 ceph-mon[29216]: pgmap v266: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:41.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:41 smithi161 ceph-mon[32709]: pgmap v266: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:43.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:43 smithi183 ceph-mon[33137]: pgmap v267: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:43.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:43 smithi102 ceph-mon[29216]: pgmap v267: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:43.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:43 smithi161 ceph-mon[32709]: pgmap v267: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 15 op/s 2024-04-24T23:21:44.361 INFO:tasks.workunit.client.0.smithi102.stdout:fs setup took 428 secs 2024-04-24T23:21:44.938 INFO:tasks.workunit.client.0.smithi102.stdout:Syncing()...0 sec 2024-04-24T23:21:44.939 INFO:tasks.workunit.client.0.smithi102.stdout:Starting Actual Benchmark At: Wed Apr 24 23:21:44 2024 2024-04-24T23:21:44.939 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:21:45.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:45 smithi102 ceph-mon[29216]: pgmap v268: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:21:45.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:45 smithi161 ceph-mon[32709]: pgmap v268: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:21:45.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:45 smithi183 ceph-mon[33137]: pgmap v268: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 37 MiB/s wr, 25 op/s 2024-04-24T23:21:47.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:47 smithi102 ceph-mon[29216]: pgmap v269: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:47.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:47 smithi161 ceph-mon[32709]: pgmap v269: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:47.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:47 smithi183 ceph-mon[33137]: pgmap v269: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 17 op/s 2024-04-24T23:21:49.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:49 smithi102 ceph-mon[29216]: pgmap v270: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 36 MiB/s wr, 1.52k op/s 2024-04-24T23:21:49.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:49 smithi161 ceph-mon[32709]: pgmap v270: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 36 MiB/s wr, 1.52k op/s 2024-04-24T23:21:49.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:49 smithi183 ceph-mon[33137]: pgmap v270: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 36 MiB/s wr, 1.52k op/s 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.277+0000 7fe3a773b640 1 -- 172.21.15.102:0/217431380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a0106630 msgr2=0x7fe3a0106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.277+0000 7fe3a773b640 1 --2- 172.21.15.102:0/217431380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a0106630 0x7fe3a0106a10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe390009cd0 tx=0x7fe39002a8f0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.277+0000 7fe3a773b640 1 -- 172.21.15.102:0/217431380 shutdown_connections 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.277+0000 7fe3a773b640 1 --2- 172.21.15.102:0/217431380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3a0101030 0x7fe3a010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.277+0000 7fe3a773b640 1 --2- 172.21.15.102:0/217431380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a0100670 0x7fe3a0100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.277+0000 7fe3a773b640 1 --2- 172.21.15.102:0/217431380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a0106630 0x7fe3a0106a10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.277+0000 7fe3a773b640 1 -- 172.21.15.102:0/217431380 >> 172.21.15.102:0/217431380 conn(0x7fe3a00777c0 msgr2=0x7fe3a0077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.277+0000 7fe3a773b640 1 -- 172.21.15.102:0/217431380 shutdown_connections 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.277+0000 7fe3a773b640 1 -- 172.21.15.102:0/217431380 wait complete. 2024-04-24T23:21:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a773b640 1 Processor -- start 2024-04-24T23:21:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a773b640 1 -- start start 2024-04-24T23:21:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a773b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a0100670 0x7fe3a0195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a773b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3a0101030 0x7fe3a0196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a773b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a0106630 0x7fe3a019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a773b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010f7e0 con 0x7fe3a0100670 2024-04-24T23:21:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a773b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010f660 con 0x7fe3a0106630 2024-04-24T23:21:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a773b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010f960 con 0x7fe3a0101030 2024-04-24T23:21:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a4cd6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3a0101030 0x7fe3a0196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a4cd6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3a0101030 0x7fe3a0196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:54294/0 (socket says 172.21.15.102:54294) 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a4cd6640 1 -- 172.21.15.102:0/300138341 learned_addr learned my addr 172.21.15.102:0/300138341 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a0106630 0x7fe3a019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.278+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a0100670 0x7fe3a0195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe3a4cd6640 1 -- 172.21.15.102:0/300138341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a0106630 msgr2=0x7fe3a019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a0106630 0x7fe3a019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe3a4cd6640 1 -- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a0100670 msgr2=0x7fe3a0195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a0100670 0x7fe3a0195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe3a4cd6640 1 -- 172.21.15.102:0/300138341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a019ad30 con 0x7fe3a0101030 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3a0101030 0x7fe3a0196280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe394010b80 tx=0x7fe39400e490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe39e7fc640 1 -- 172.21.15.102:0/300138341 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39401a030 con 0x7fe3a0101030 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe3a773b640 1 -- 172.21.15.102:0/300138341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a019b020 con 0x7fe3a0101030 2024-04-24T23:21:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe39e7fc640 1 -- 172.21.15.102:0/300138341 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe394004020 con 0x7fe3a0101030 2024-04-24T23:21:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe39e7fc640 1 -- 172.21.15.102:0/300138341 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394003c40 con 0x7fe3a0101030 2024-04-24T23:21:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.279+0000 7fe3a773b640 1 -- 172.21.15.102:0/300138341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a01a28b0 con 0x7fe3a0101030 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.282+0000 7fe39e7fc640 1 -- 172.21.15.102:0/300138341 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe39401d070 con 0x7fe3a0101030 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.282+0000 7fe3aa449740 1 Processor -- start 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.282+0000 7fe3aa449740 1 -- start start 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe39e7fc640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe3880a2da0 0x7fe3880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cf9f10 0x5629c8cfa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cfa830 0x5629c8d09100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d0ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3aa449740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5629c8c5ecb0 con 0x5629c8d09640 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3aa449740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5629c8c075b0 con 0x5629c8cf9f10 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3aa449740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5629c8c5ac70 con 0x5629c8cfa830 2024-04-24T23:21:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe39e7fc640 1 -- 172.21.15.102:0/300138341 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7fe3940d1090 con 0x7fe3a0101030 2024-04-24T23:21:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe3880a2da0 0x7fe3880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3a5cd8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d0ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3a5cd8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d0ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36192/0 (socket says 172.21.15.102:36192) 2024-04-24T23:21:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3a5cd8640 1 -- 172.21.15.102:0/2404423077 learned_addr learned my addr 172.21.15.102:0/2404423077 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cf9f10 0x5629c8cfa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.283+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cfa830 0x5629c8d09100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3a5cd8640 1 -- 172.21.15.102:0/2404423077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cfa830 msgr2=0x5629c8d09100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cfa830 0x5629c8d09100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3a5cd8640 1 -- 172.21.15.102:0/2404423077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cf9f10 msgr2=0x5629c8cfa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cf9f10 0x5629c8cfa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3a5cd8640 1 -- 172.21.15.102:0/2404423077 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c8973210 con 0x5629c8d09640 2024-04-24T23:21:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cf9f10 0x5629c8cfa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe3880a2da0 0x7fe3880a5260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe398009fd0 tx=0x7fe398009b80 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d0ba30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe398005a70 tx=0x7fe39801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424649949ff98d01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3877fe640 1 -- 172.21.15.102:0/2404423077 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398024070 con 0x5629c8d09640 2024-04-24T23:21:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3877fe640 1 -- 172.21.15.102:0/2404423077 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe39800fe50 con 0x5629c8d09640 2024-04-24T23:21:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.284+0000 7fe3877fe640 1 -- 172.21.15.102:0/2404423077 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39801f940 con 0x5629c8d09640 2024-04-24T23:21:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.285+0000 7fe3aa449740 1 -- 172.21.15.102:0/2404423077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 msgr2=0x5629c8d0ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.285+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d0ba30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe398005a70 tx=0x7fe39801d000 comp rx=0 tx=0).stop 2024-04-24T23:21:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.287+0000 7fe3aa449740 1 -- 172.21.15.102:0/2404423077 shutdown_connections 2024-04-24T23:21:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.287+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d0ba30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.287+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cfa830 0x5629c8d09100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.287+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2404423077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cf9f10 0x5629c8cfa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.287+0000 7fe3aa449740 1 -- 172.21.15.102:0/2404423077 >> 172.21.15.102:0/2404423077 conn(0x5629c8cf6560 msgr2=0x5629c8cf6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.287+0000 7fe3aa449740 1 -- 172.21.15.102:0/2404423077 shutdown_connections 2024-04-24T23:21:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.287+0000 7fe3aa449740 1 -- 172.21.15.102:0/2404423077 wait complete. 2024-04-24T23:21:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.288+0000 7fe3aa449740 1 -- start start 2024-04-24T23:21:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.288+0000 7fe3aa449740 1 Processor -- start 2024-04-24T23:21:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d08f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d02250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d02790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3aa449740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5629c8d0c3a0 con 0x5629c8d09640 2024-04-24T23:21:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3aa449740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5629c8c1ab50 con 0x5629c8cfa830 2024-04-24T23:21:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3aa449740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5629c8d0c520 con 0x5629c8cf9f10 2024-04-24T23:21:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3aa449740 10 client.0 _ll_register_callbacks cb 0x5629c8c55ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3a4cd6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d02250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3a54d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d08f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3a4cd6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d02250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:46006/0 (socket says 172.21.15.102:46006) 2024-04-24T23:21:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3a4cd6640 1 -- 172.21.15.102:0/2817174017 learned_addr learned my addr 172.21.15.102:0/2817174017 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.289+0000 7fe3a54d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d08f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:54306/0 (socket says 172.21.15.102:54306) 2024-04-24T23:21:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d02790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3a4cd6640 1 -- 172.21.15.102:0/2817174017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 msgr2=0x5629c8d08f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d08f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3a4cd6640 1 -- 172.21.15.102:0/2817174017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 msgr2=0x5629c8d02790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d02790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3a4cd6640 1 -- 172.21.15.102:0/2817174017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c8a3f620 con 0x5629c8cfa830 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d08f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d02790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d02250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe394099970 tx=0x7fe39409dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec91ea9b0d3ea50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe384ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe384ff9640 1 -- 172.21.15.102:0/2817174017 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3940a6180 con 0x5629c8cfa830 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe384ff9640 1 -- 172.21.15.102:0/2817174017 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7fe3940041d0 con 0x5629c8cfa830 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.290+0000 7fe3aa449740 1 -- 172.21.15.102:0/2817174017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe39800fe50 con 0x5629c8cfa830 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.291+0000 7fe384ff9640 1 -- 172.21.15.102:0/2817174017 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe39400eb40 con 0x5629c8cfa830 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.291+0000 7fe3aa449740 10 client.25508 fetch_fsmap learned FSMap version 26 2024-04-24T23:21:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.291+0000 7fe3aa449740 1 -- 172.21.15.102:0/2817174017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629c8d1caa0 con 0x5629c8cfa830 2024-04-24T23:21:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.291+0000 7fe384ff9640 1 -- 172.21.15.102:0/2817174017 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe39800fe50 con 0x5629c8cfa830 2024-04-24T23:21:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.292+0000 7fe384ff9640 1 -- 172.21.15.102:0/2817174017 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe3940993f0 con 0x5629c8cfa830 2024-04-24T23:21:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.292+0000 7fe3aa449740 10 client.25508 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:21:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.292+0000 7fe3aa449740 10 client.25508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:21:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.292+0000 7fe3aa449740 20 client.25508 populate_metadata read hostname 'smithi102' 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.292+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5629c8d1df10 0x5629c8d20300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.292+0000 7fe3aa449740 4 client.25508 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.292+0000 7fe3aa449740 1 -- 172.21.15.102:0/2817174017 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c8c6c1f0 con 0x5629c8d1df10 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.292+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5629c8d1df10 0x5629c8d20300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.293+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5629c8d1df10 0x5629c8d20300 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b8da65e1346a10 server_cookie=889ea554ac785f9b in_seq=0 out_seq=0 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.293+0000 7fe384ff9640 10 client.25508 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.293+0000 7fe384ff9640 1 -- 172.21.15.102:0/2817174017 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629c8c6c1f0 con 0x5629c8d1df10 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.293+0000 7fe384ff9640 10 client.25508 handle_command_reply: tid=0 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 client.25508 shutdown 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 -- 172.21.15.102:0/2817174017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 msgr2=0x5629c8d02250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d02250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe394099970 tx=0x7fe39409dc10 comp rx=0 tx=0).stop 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 -- 172.21.15.102:0/2817174017 shutdown_connections 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5629c8d1df10 0x5629c8d20300 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d02790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d02250 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 --2- 172.21.15.102:0/2817174017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d08f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 -- 172.21.15.102:0/2817174017 >> 172.21.15.102:0/2817174017 conn(0x5629c8cf6560 msgr2=0x5629c8cfc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe384ff9640 0 client.25508 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 -- 172.21.15.102:0/2817174017 shutdown_connections 2024-04-24T23:21:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 1 -- 172.21.15.102:0/2817174017 wait complete. 2024-04-24T23:21:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.294+0000 7fe3aa449740 20 client.25508 trim_cache size 0 max 16384 2024-04-24T23:21:51.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:51 smithi102 ceph-mon[29216]: pgmap v271: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 1.52k op/s 2024-04-24T23:21:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.321+0000 7fe3aa449740 1 Processor -- start 2024-04-24T23:21:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.321+0000 7fe3aa449740 1 -- start start 2024-04-24T23:21:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8cfc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8cfa830 0x5629c8d08a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3a54d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8cfc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3a4cd6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8cfa830 0x5629c8d08a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3a54d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8cfc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:54316/0 (socket says 172.21.15.102:54316) 2024-04-24T23:21:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3a4cd6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8cfa830 0x5629c8d08a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36204/0 (socket says 172.21.15.102:36204) 2024-04-24T23:21:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3a54d7640 1 -- 172.21.15.102:0/464471731 learned_addr learned my addr 172.21.15.102:0/464471731 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3aa449740 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8d09640 0x5629c8d25bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3aa449740 1 -- 172.21.15.102:0/464471731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01adc80 con 0x5629c8cfa830 2024-04-24T23:21:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3aa449740 1 -- 172.21.15.102:0/464471731 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01ab550 con 0x5629c8d09640 2024-04-24T23:21:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3aa449740 1 -- 172.21.15.102:0/464471731 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01a93c0 con 0x5629c8cf9f10 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.322+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8d09640 0x5629c8d25bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe3a54d7640 1 -- 172.21.15.102:0/464471731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8d09640 msgr2=0x5629c8d25bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8d09640 0x5629c8d25bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe3a54d7640 1 -- 172.21.15.102:0/464471731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8cfa830 msgr2=0x5629c8d08a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8cfa830 0x5629c8d08a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe3a54d7640 1 -- 172.21.15.102:0/464471731 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c8d031b0 con 0x5629c8cf9f10 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8cfa830 0x5629c8d08a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8cfc920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe390009cd0 tx=0x7fe390009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398b31775c8d315e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe386ffd640 1 -- 172.21.15.102:0/464471731 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390003970 con 0x5629c8cf9f10 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe386ffd640 1 -- 172.21.15.102:0/464471731 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe39004e070 con 0x5629c8cf9f10 2024-04-24T23:21:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.323+0000 7fe386ffd640 1 -- 172.21.15.102:0/464471731 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39004a0f0 con 0x5629c8cf9f10 2024-04-24T23:21:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.324+0000 7fe3aa449740 1 -- 172.21.15.102:0/464471731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 msgr2=0x5629c8cfc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.325+0000 7fe3aa449740 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8cfc920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe390009cd0 tx=0x7fe390009360 comp rx=0 tx=0).stop 2024-04-24T23:21:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.325+0000 7fe3aa449740 1 -- 172.21.15.102:0/464471731 shutdown_connections 2024-04-24T23:21:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.325+0000 7fe3aa449740 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8d09640 0x5629c8d25bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.325+0000 7fe3aa449740 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8cfa830 0x5629c8d08a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.325+0000 7fe3aa449740 1 --2- 172.21.15.102:0/464471731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8cfc920 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.325+0000 7fe3aa449740 1 -- 172.21.15.102:0/464471731 >> 172.21.15.102:0/464471731 conn(0x5629c8cf6560 msgr2=0x5629c8cfd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.325+0000 7fe3aa449740 1 -- 172.21.15.102:0/464471731 shutdown_connections 2024-04-24T23:21:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.325+0000 7fe3aa449740 1 -- 172.21.15.102:0/464471731 wait complete. 2024-04-24T23:21:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.326+0000 7fe3aa449740 1 -- start start 2024-04-24T23:21:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.326+0000 7fe3aa449740 1 Processor -- start 2024-04-24T23:21:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.327+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d7f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.327+0000 7fe3aa449740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d7f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.327+0000 7fe3a54d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d7f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.328+0000 7fe3a54d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d7f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:54318/0 (socket says 172.21.15.102:54318) 2024-04-24T23:21:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.328+0000 7fe3a54d7640 1 -- 172.21.15.102:0/3095660301 learned_addr learned my addr 172.21.15.102:0/3095660301 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:21:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.328+0000 7fe3aa449740 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d7fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.328+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d7f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.328+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01a9a10 con 0x5629c8d09640 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.328+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01aae80 con 0x5629c8cfa830 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.328+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01aa0e0 con 0x5629c8cf9f10 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.328+0000 7fe3aa449740 10 client.0 _ll_register_callbacks cb 0x7fe394099840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.329+0000 7fe3a5cd8640 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d7fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.329+0000 7fe3a54d7640 1 -- 172.21.15.102:0/3095660301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 msgr2=0x5629c8d7f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.329+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d7f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.329+0000 7fe3a54d7640 1 -- 172.21.15.102:0/3095660301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 msgr2=0x5629c8d7fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.329+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d7fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.329+0000 7fe3a54d7640 1 -- 172.21.15.102:0/3095660301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c8cfe920 con 0x5629c8cf9f10 2024-04-24T23:21:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.329+0000 7fe3a54d7640 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d7f420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe39002ae10 tx=0x7fe390030e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604bcad49eb509f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.329+0000 7fe384ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:21:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.330+0000 7fe384ff9640 1 -- 172.21.15.102:0/3095660301 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390004250 con 0x5629c8cf9f10 2024-04-24T23:21:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.330+0000 7fe384ff9640 1 -- 172.21.15.102:0/3095660301 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7fe390004b40 con 0x5629c8cf9f10 2024-04-24T23:21:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.330+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe39004e070 con 0x5629c8cf9f10 2024-04-24T23:21:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.330+0000 7fe384ff9640 1 -- 172.21.15.102:0/3095660301 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe390063050 con 0x5629c8cf9f10 2024-04-24T23:21:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.330+0000 7fe3aa449740 10 client.25441 fetch_fsmap learned FSMap version 26 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.331+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629c8d15cd0 con 0x5629c8cf9f10 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.331+0000 7fe384ff9640 1 -- 172.21.15.102:0/3095660301 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe390036040 con 0x5629c8cf9f10 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.331+0000 7fe384ff9640 1 -- 172.21.15.102:0/3095660301 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe39004a0f0 con 0x5629c8cf9f10 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.331+0000 7fe3aa449740 10 client.25441 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.331+0000 7fe3aa449740 10 client.25441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.331+0000 7fe3aa449740 20 client.25441 populate_metadata read hostname 'smithi102' 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.331+0000 7fe3aa449740 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5629c8d16380 0x5629c8d187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.331+0000 7fe3aa449740 4 client.25441 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.331+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe39004e070 con 0x5629c8d16380 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.332+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5629c8d16380 0x5629c8d187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.332+0000 7fe3a4cd6640 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5629c8d16380 0x5629c8d187a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=647582352b9a719c server_cookie=7eb44ee4647f85c3 in_seq=0 out_seq=0 2024-04-24T23:21:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.332+0000 7fe384ff9640 10 client.25441 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.342+0000 7fe384ff9640 1 -- 172.21.15.102:0/3095660301 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe39004e070 con 0x5629c8d16380 2024-04-24T23:21:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.342+0000 7fe384ff9640 10 client.25441 handle_command_reply: tid=0 2024-04-24T23:21:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.342+0000 7fe3aa449740 1 client.25441 shutdown 2024-04-24T23:21:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 msgr2=0x5629c8d7f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d7f420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe39002ae10 tx=0x7fe390030e60 comp rx=0 tx=0).stop 2024-04-24T23:21:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 shutdown_connections 2024-04-24T23:21:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5629c8d16380 0x5629c8d187a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629c8d09640 0x5629c8d7fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe384ff9640 0 client.25441 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:21:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5629c8cfa830 0x5629c8d7f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 --2- 172.21.15.102:0/3095660301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629c8cf9f10 0x5629c8d7f420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 >> 172.21.15.102:0/3095660301 conn(0x5629c8cf6560 msgr2=0x5629c8d00ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 shutdown_connections 2024-04-24T23:21:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 1 -- 172.21.15.102:0/3095660301 wait complete. 2024-04-24T23:21:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.343+0000 7fe3aa449740 20 client.25441 trim_cache size 0 max 16384 2024-04-24T23:21:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:21:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:21:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:21:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:21:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.344+0000 7fe3a773b640 1 -- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe3880a2da0 msgr2=0x7fe3880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.344+0000 7fe3a773b640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe3880a2da0 0x7fe3880a5260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe398009fd0 tx=0x7fe398009b80 comp rx=0 tx=0).stop 2024-04-24T23:21:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.344+0000 7fe3a773b640 1 -- 172.21.15.102:0/300138341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3a0101030 msgr2=0x7fe3a0196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.344+0000 7fe3a773b640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3a0101030 0x7fe3a0196280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe394010b80 tx=0x7fe39400e490 comp rx=0 tx=0).stop 2024-04-24T23:21:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.344+0000 7fe3a773b640 1 -- 172.21.15.102:0/300138341 shutdown_connections 2024-04-24T23:21:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.344+0000 7fe3a773b640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3a0106630 0x7fe3a019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.344+0000 7fe3a773b640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3a0101030 0x7fe3a0196280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.344+0000 7fe3a773b640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe3880a2da0 0x7fe3880a5260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.344+0000 7fe3a773b640 1 --2- 172.21.15.102:0/300138341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a0100670 0x7fe3a0195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.345+0000 7fe3a773b640 1 -- 172.21.15.102:0/300138341 >> 172.21.15.102:0/300138341 conn(0x7fe3a00777c0 msgr2=0x7fe3a01032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.345+0000 7fe3a773b640 1 -- 172.21.15.102:0/300138341 shutdown_connections 2024-04-24T23:21:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:21:51.345+0000 7fe3a773b640 1 -- 172.21.15.102:0/300138341 wait complete. 2024-04-24T23:21:51.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:51 smithi161 ceph-mon[32709]: pgmap v271: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 1.52k op/s 2024-04-24T23:21:51.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:51 smithi183 ceph-mon[33137]: pgmap v271: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 1.52k op/s 2024-04-24T23:21:53.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:53 smithi102 ceph-mon[29216]: pgmap v272: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 1.52k op/s 2024-04-24T23:21:53.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:53 smithi161 ceph-mon[32709]: pgmap v272: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 1.52k op/s 2024-04-24T23:21:53.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:53 smithi183 ceph-mon[33137]: pgmap v272: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 1.52k op/s 2024-04-24T23:21:55.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:55 smithi102 ceph-mon[29216]: pgmap v273: 97 pgs: 97 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 35 MiB/s wr, 4.64k op/s 2024-04-24T23:21:55.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:55 smithi161 ceph-mon[32709]: pgmap v273: 97 pgs: 97 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 35 MiB/s wr, 4.64k op/s 2024-04-24T23:21:55.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:55 smithi183 ceph-mon[33137]: pgmap v273: 97 pgs: 97 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 35 MiB/s wr, 4.64k op/s 2024-04-24T23:21:57.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:57 smithi102 ceph-mon[29216]: pgmap v274: 97 pgs: 97 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 4.63k op/s 2024-04-24T23:21:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:57 smithi161 ceph-mon[32709]: pgmap v274: 97 pgs: 97 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 4.63k op/s 2024-04-24T23:21:57.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:57 smithi183 ceph-mon[33137]: pgmap v274: 97 pgs: 97 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 4.63k op/s 2024-04-24T23:21:59.313 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:21:59 smithi183 ceph-mon[33137]: pgmap v275: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 7.78k op/s 2024-04-24T23:21:59.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:21:59 smithi102 ceph-mon[29216]: pgmap v275: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 7.78k op/s 2024-04-24T23:21:59.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:21:59 smithi161 ceph-mon[32709]: pgmap v275: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 7.78k op/s 2024-04-24T23:22:01.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:01 smithi102 ceph-mon[29216]: pgmap v276: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:22:01.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:01 smithi161 ceph-mon[32709]: pgmap v276: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:22:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:01 smithi183 ceph-mon[33137]: pgmap v276: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:22:03.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:03 smithi102 ceph-mon[29216]: pgmap v277: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:22:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:03 smithi161 ceph-mon[32709]: pgmap v277: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:22:03.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:03 smithi183 ceph-mon[33137]: pgmap v277: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:22:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.501+0000 7fba05053640 1 -- 172.21.15.102:0/801195005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba001007f0 msgr2=0x7fba00100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.502+0000 7fba05053640 1 --2- 172.21.15.102:0/801195005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba001007f0 0x7fba00100bd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009ca0 tx=0x7fb9f802a9e0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.504+0000 7fba05053640 1 -- 172.21.15.102:0/801195005 shutdown_connections 2024-04-24T23:22:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.504+0000 7fba05053640 1 --2- 172.21.15.102:0/801195005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba00105b10 0x7fba00105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.504+0000 7fba05053640 1 --2- 172.21.15.102:0/801195005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba00101110 0x7fba001055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.504+0000 7fba05053640 1 --2- 172.21.15.102:0/801195005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba001007f0 0x7fba00100bd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.504+0000 7fba05053640 1 -- 172.21.15.102:0/801195005 >> 172.21.15.102:0/801195005 conn(0x7fba000769c0 msgr2=0x7fba00076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.505+0000 7fba05053640 1 -- 172.21.15.102:0/801195005 shutdown_connections 2024-04-24T23:22:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.505+0000 7fba05053640 1 -- 172.21.15.102:0/801195005 wait complete. 2024-04-24T23:22:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.506+0000 7fba05053640 1 Processor -- start 2024-04-24T23:22:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.506+0000 7fba05053640 1 -- start start 2024-04-24T23:22:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fba05053640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba001007f0 0x7fba0019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fba05053640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba00101110 0x7fba0019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fba05053640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba00105b10 0x7fba001a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fba05053640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba00118010 con 0x7fba00101110 2024-04-24T23:22:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fba05053640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fba00117e90 con 0x7fba00105b10 2024-04-24T23:22:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fba05053640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba00118190 con 0x7fba001007f0 2024-04-24T23:22:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9fed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba001007f0 0x7fba0019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9fe59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba00101110 0x7fba0019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9fed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba001007f0 0x7fba0019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:47860/0 (socket says 172.21.15.102:47860) 2024-04-24T23:22:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9fed9d640 1 -- 172.21.15.102:0/774048293 learned_addr learned my addr 172.21.15.102:0/774048293 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9ff59e640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba00105b10 0x7fba001a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9fed9d640 1 -- 172.21.15.102:0/774048293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba00105b10 msgr2=0x7fba001a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba00105b10 0x7fba001a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9fed9d640 1 -- 172.21.15.102:0/774048293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba00101110 msgr2=0x7fba0019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba00101110 0x7fba0019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.507+0000 7fb9fed9d640 1 -- 172.21.15.102:0/774048293 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba001a3450 con 0x7fba001007f0 2024-04-24T23:22:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.508+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba001007f0 0x7fba0019e4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9f802afd0 tx=0x7fb9f802fbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.511+0000 7fb9e7fff640 1 -- 172.21.15.102:0/774048293 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8033040 con 0x7fba001007f0 2024-04-24T23:22:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.511+0000 7fb9e7fff640 1 -- 172.21.15.102:0/774048293 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9f802e7a0 con 0x7fba001007f0 2024-04-24T23:22:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.511+0000 7fb9e7fff640 1 -- 172.21.15.102:0/774048293 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8043720 con 0x7fba001007f0 2024-04-24T23:22:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.514+0000 7fba05053640 1 -- 172.21.15.102:0/774048293 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba001a36e0 con 0x7fba001007f0 2024-04-24T23:22:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.514+0000 7fba05053640 1 -- 172.21.15.102:0/774048293 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001a3af0 con 0x7fba001007f0 2024-04-24T23:22:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fb9e7fff640 1 -- 172.21.15.102:0/774048293 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fb9f8036070 con 0x7fba001007f0 2024-04-24T23:22:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fba07e34740 1 Processor -- start 2024-04-24T23:22:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fba07e34740 1 -- start start 2024-04-24T23:22:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fba07e34740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64fe0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fba07e34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0b60 0x555e64feacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fba07e34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64feb230 0x555e64fed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fba07e34740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555e64f44cb0 con 0x555e64fe0240 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fba07e34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555e64eed5b0 con 0x555e64fe0b60 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fba07e34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e64f40c70 con 0x555e64feb230 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fb9fe59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64feb230 0x555e64fed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fb9fe59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64feb230 0x555e64fed620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:47862/0 (socket says 172.21.15.102:47862) 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fb9fe59c640 1 -- 172.21.15.102:0/3907569366 learned_addr learned my addr 172.21.15.102:0/3907569366 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fb9ff59e640 1 --2- 172.21.15.102:0/3907569366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64fe0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.518+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/3907569366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0b60 0x555e64feacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9e7fff640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9e80a2f80 0x7fb9e80a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9fe59c640 1 -- 172.21.15.102:0/3907569366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0b60 msgr2=0x555e64feacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/3907569366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0b60 0x555e64feacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9fe59c640 1 -- 172.21.15.102:0/3907569366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 msgr2=0x555e64fe0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/3907569366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64fe0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9fe59c640 1 -- 172.21.15.102:0/3907569366 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e64c59210 con 0x555e64feb230 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9ff59e640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9e80a2f80 0x7fb9e80a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9e7fff640 1 -- 172.21.15.102:0/774048293 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7fb9f80f0010 con 0x7fba001007f0 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/3907569366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64feb230 0x555e64fed620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000bcb0 tx=0x7fb9f00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df1a43966e310257 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9e57fa640 1 -- 172.21.15.102:0/3907569366 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0004030 con 0x555e64feb230 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9e57fa640 1 -- 172.21.15.102:0/3907569366 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9f00041d0 con 0x555e64feb230 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.519+0000 7fb9e57fa640 1 -- 172.21.15.102:0/3907569366 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f00116c0 con 0x555e64feb230 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fba07e34740 1 -- 172.21.15.102:0/3907569366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64feb230 msgr2=0x555e64fed620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fb9ff59e640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9e80a2f80 0x7fb9e80a5440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40058c0 tx=0x7fb9f40046c0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fba07e34740 1 --2- 172.21.15.102:0/3907569366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64feb230 0x555e64fed620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000bcb0 tx=0x7fb9f00107a0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fba07e34740 1 -- 172.21.15.102:0/3907569366 shutdown_connections 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fba07e34740 1 --2- 172.21.15.102:0/3907569366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64feb230 0x555e64fed620 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fba07e34740 1 --2- 172.21.15.102:0/3907569366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0b60 0x555e64feacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fba07e34740 1 --2- 172.21.15.102:0/3907569366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64fe0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fba07e34740 1 -- 172.21.15.102:0/3907569366 >> 172.21.15.102:0/3907569366 conn(0x555e64fdc560 msgr2=0x555e64fdc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fba07e34740 1 -- 172.21.15.102:0/3907569366 shutdown_connections 2024-04-24T23:22:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.520+0000 7fba07e34740 1 -- 172.21.15.102:0/3907569366 wait complete. 2024-04-24T23:22:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fba07e34740 1 -- start start 2024-04-24T23:22:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fba07e34740 1 Processor -- start 2024-04-24T23:22:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fba07e34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64fe81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fba07e34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fe8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fba07e34740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 0x555e64fe8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fba07e34740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555e64fee020 con 0x555e64feb230 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fba07e34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555e64f00b50 con 0x555e64fe0240 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fba07e34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e64fee1a0 con 0x555e64fe0b60 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fb9fe59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fe8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fb9fe59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fe8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:47864/0 (socket says 172.21.15.102:47864) 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fb9fe59c640 1 -- 172.21.15.102:0/3283622232 learned_addr learned my addr 172.21.15.102:0/3283622232 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fba07e34740 10 client.0 _ll_register_callbacks cb 0x555e64f3bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.521+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64fe81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9ff59e640 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 0x555e64fe8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9fe59c640 1 -- 172.21.15.102:0/3283622232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 msgr2=0x555e64fe81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64fe81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9fe59c640 1 -- 172.21.15.102:0/3283622232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 msgr2=0x555e64fe8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 0x555e64fe8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9fe59c640 1 -- 172.21.15.102:0/3283622232 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e64d25620 con 0x555e64fe0b60 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fe8730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00045e0 tx=0x7fb9f0004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3295edd5580bb767 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/3283622232 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0003a00 con 0x555e64fe0b60 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/3283622232 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7fb9f0005040 con 0x555e64fe0b60 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fba07e34740 1 -- 172.21.15.102:0/3283622232 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f00041d0 con 0x555e64fe0b60 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/3283622232 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9f0032050 con 0x555e64fe0b60 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fba07e34740 10 client.25453 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.522+0000 7fba07e34740 1 -- 172.21.15.102:0/3283622232 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e64ffa050 con 0x555e64fe0b60 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.523+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/3283622232 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9f00041d0 con 0x555e64fe0b60 2024-04-24T23:22:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.523+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/3283622232 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb9f0004a70 con 0x555e64fe0b60 2024-04-24T23:22:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.523+0000 7fba07e34740 10 client.25453 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.523+0000 7fba07e34740 10 client.25453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.523+0000 7fba07e34740 20 client.25453 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.523+0000 7fba07e34740 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555e64ffb400 0x555e64ffd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.523+0000 7fba07e34740 4 client.25453 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.523+0000 7fba07e34740 1 -- 172.21.15.102:0/3283622232 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e64f521f0 con 0x555e64ffb400 2024-04-24T23:22:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.523+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555e64ffb400 0x555e64ffd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555e64ffb400 0x555e64ffd7f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e0adc497e01b1e server_cookie=72b821db628ed9de in_seq=0 out_seq=0 2024-04-24T23:22:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fb9e2ff5640 10 client.25453 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/3283622232 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e64f521f0 con 0x555e64ffb400 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fb9e2ff5640 10 client.25453 handle_command_reply: tid=0 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fba07e34740 1 client.25453 shutdown 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fba07e34740 1 -- 172.21.15.102:0/3283622232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 msgr2=0x555e64fe8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fba07e34740 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fe8730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00045e0 tx=0x7fb9f0004610 comp rx=0 tx=0).stop 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fba07e34740 1 -- 172.21.15.102:0/3283622232 shutdown_connections 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fba07e34740 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555e64ffb400 0x555e64ffd7f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fba07e34740 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 0x555e64fe8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fb9e2ff5640 0 client.25453 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fba07e34740 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fe8730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fba07e34740 1 --2- 172.21.15.102:0/3283622232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64fe81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.524+0000 7fba07e34740 1 -- 172.21.15.102:0/3283622232 >> 172.21.15.102:0/3283622232 conn(0x555e64fdc560 msgr2=0x555e64fe2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.525+0000 7fba07e34740 1 -- 172.21.15.102:0/3283622232 shutdown_connections 2024-04-24T23:22:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.525+0000 7fba07e34740 1 -- 172.21.15.102:0/3283622232 wait complete. 2024-04-24T23:22:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.525+0000 7fba07e34740 20 client.25453 trim_cache size 0 max 16384 2024-04-24T23:22:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.552+0000 7fba07e34740 1 Processor -- start 2024-04-24T23:22:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fba07e34740 1 -- start start 2024-04-24T23:22:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fba07e34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64ff6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fba07e34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fdc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fba07e34740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 0x555e64fdbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fba07e34740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b6280 con 0x555e64feb230 2024-04-24T23:22:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fba07e34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b3b50 con 0x555e64fe0240 2024-04-24T23:22:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fba07e34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b19c0 con 0x555e64fe0b60 2024-04-24T23:22:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fb9fe59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fdc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fb9fed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64ff6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fb9fed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64ff6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:55434/0 (socket says 172.21.15.102:55434) 2024-04-24T23:22:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.553+0000 7fb9fed9d640 1 -- 172.21.15.102:0/2669438822 learned_addr learned my addr 172.21.15.102:0/2669438822 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.554+0000 7fb9ff59e640 1 --2- 172.21.15.102:0/2669438822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 0x555e64fdbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.554+0000 7fb9fed9d640 1 -- 172.21.15.102:0/2669438822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 msgr2=0x555e64fdc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.554+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/2669438822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fdc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.554+0000 7fb9fed9d640 1 -- 172.21.15.102:0/2669438822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 msgr2=0x555e64fdbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.554+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/2669438822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 0x555e64fdbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.554+0000 7fb9fed9d640 1 -- 172.21.15.102:0/2669438822 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e64fe71a0 con 0x555e64fe0240 2024-04-24T23:22:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.554+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/2669438822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64ff6ee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8004af0 tx=0x7fb9f80ef040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c7763739267dcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.555+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/2669438822 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80cbe60 con 0x555e64fe0240 2024-04-24T23:22:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.555+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/2669438822 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9f80d0070 con 0x555e64fe0240 2024-04-24T23:22:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.555+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/2669438822 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80ba840 con 0x555e64fe0240 2024-04-24T23:22:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.556+0000 7fba07e34740 1 -- 172.21.15.102:0/2669438822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 msgr2=0x555e64ff6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.556+0000 7fba07e34740 1 --2- 172.21.15.102:0/2669438822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64ff6ee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8004af0 tx=0x7fb9f80ef040 comp rx=0 tx=0).stop 2024-04-24T23:22:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.556+0000 7fba07e34740 1 -- 172.21.15.102:0/2669438822 shutdown_connections 2024-04-24T23:22:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.556+0000 7fba07e34740 1 --2- 172.21.15.102:0/2669438822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64feb230 0x555e64fdbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.556+0000 7fba07e34740 1 --2- 172.21.15.102:0/2669438822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fdc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.556+0000 7fba07e34740 1 --2- 172.21.15.102:0/2669438822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64fe0240 0x555e64ff6ee0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.556+0000 7fba07e34740 1 -- 172.21.15.102:0/2669438822 >> 172.21.15.102:0/2669438822 conn(0x555e64fdc560 msgr2=0x555e64fe2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 -- 172.21.15.102:0/2669438822 shutdown_connections 2024-04-24T23:22:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 -- 172.21.15.102:0/2669438822 wait complete. 2024-04-24T23:22:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 -- start start 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 Processor -- start 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64feeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64feb230 0x555e64fef950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b2010 con 0x555e64fe0240 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b3480 con 0x555e64feb230 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.557+0000 7fba07e34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b26e0 con 0x555e64fe0b60 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fba07e34740 10 client.0 _ll_register_callbacks cb 0x7fb9f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fed9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64feeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fed9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64feeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57128/0 (socket says 172.21.15.102:57128) 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fed9d640 1 -- 172.21.15.102:0/1810672578 learned_addr learned my addr 172.21.15.102:0/1810672578 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9ff59e640 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64feb230 0x555e64fef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fe59c640 1 -- 172.21.15.102:0/1810672578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64feb230 msgr2=0x555e64fef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64feb230 0x555e64fef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fe59c640 1 -- 172.21.15.102:0/1810672578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 msgr2=0x555e64feeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64feeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fe59c640 1 -- 172.21.15.102:0/1810672578 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e64fed990 con 0x555e64fe0b60 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64feeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9fe59c640 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fef410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0011490 tx=0x7fb9f0003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6091e678131db1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/1810672578 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f003a070 con 0x555e64fe0b60 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/1810672578 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8579+0+0 (secure 0 0 0) 0x7fb9f0003800 con 0x555e64fe0b60 2024-04-24T23:22:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.558+0000 7fba07e34740 1 -- 172.21.15.102:0/1810672578 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f80d0070 con 0x555e64fe0b60 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/1810672578 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9f0046050 con 0x555e64fe0b60 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fba07e34740 10 client.25459 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fba07e34740 1 -- 172.21.15.102:0/1810672578 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e65061ab0 con 0x555e64fe0b60 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/1810672578 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9f0029070 con 0x555e64fe0b60 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/1810672578 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb9f0004680 con 0x555e64fe0b60 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fba07e34740 10 client.25459 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fba07e34740 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fba07e34740 20 client.25459 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fba07e34740 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555e65062200 0x555e65064620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fba07e34740 4 client.25459 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fba07e34740 1 -- 172.21.15.102:0/1810672578 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb9f80d0070 con 0x555e65062200 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.559+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555e65062200 0x555e65064620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.560+0000 7fb9e2ff5640 10 client.25459 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.560+0000 7fb9fed9d640 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555e65062200 0x555e65064620 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fef37a2a4a0edb7 server_cookie=925345aa68bac46c in_seq=0 out_seq=0 2024-04-24T23:22:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fb9e2ff5640 1 -- 172.21.15.102:0/1810672578 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fb9f80d0070 con 0x555e65062200 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fb9e2ff5640 10 client.25459 handle_command_reply: tid=0 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fba07e34740 1 client.25459 shutdown 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fba07e34740 1 -- 172.21.15.102:0/1810672578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 msgr2=0x555e64fef410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fba07e34740 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fef410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0011490 tx=0x7fb9f0003ed0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fba07e34740 1 -- 172.21.15.102:0/1810672578 shutdown_connections 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fba07e34740 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555e64feb230 0x555e64fef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fba07e34740 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e64fe0b60 0x555e64fef410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fba07e34740 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555e65062200 0x555e65064620 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fba07e34740 1 --2- 172.21.15.102:0/1810672578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fe0240 0x555e64feeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fba07e34740 1 -- 172.21.15.102:0/1810672578 >> 172.21.15.102:0/1810672578 conn(0x555e64fdc560 msgr2=0x555e64fe2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.567+0000 7fb9e2ff5640 0 client.25459 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.568+0000 7fba07e34740 1 -- 172.21.15.102:0/1810672578 shutdown_connections 2024-04-24T23:22:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.568+0000 7fba07e34740 1 -- 172.21.15.102:0/1810672578 wait complete. 2024-04-24T23:22:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.568+0000 7fba07e34740 20 client.25459 trim_cache size 0 max 16384 2024-04-24T23:22:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:22:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:22:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:22:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:22:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.569+0000 7fba05053640 1 -- 172.21.15.102:0/774048293 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9e80a2f80 msgr2=0x7fb9e80a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.569+0000 7fba05053640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9e80a2f80 0x7fb9e80a5440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40058c0 tx=0x7fb9f40046c0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.569+0000 7fba05053640 1 -- 172.21.15.102:0/774048293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba001007f0 msgr2=0x7fba0019e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.569+0000 7fba05053640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba001007f0 0x7fba0019e4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9f802afd0 tx=0x7fb9f802fbe0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.570+0000 7fba05053640 1 -- 172.21.15.102:0/774048293 shutdown_connections 2024-04-24T23:22:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.570+0000 7fba05053640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba00105b10 0x7fba001a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.570+0000 7fba05053640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fb9e80a2f80 0x7fb9e80a5440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.570+0000 7fba05053640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba00101110 0x7fba0019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.570+0000 7fba05053640 1 --2- 172.21.15.102:0/774048293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba001007f0 0x7fba0019e4b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.570+0000 7fba05053640 1 -- 172.21.15.102:0/774048293 >> 172.21.15.102:0/774048293 conn(0x7fba000769c0 msgr2=0x7fba001035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.570+0000 7fba05053640 1 -- 172.21.15.102:0/774048293 shutdown_connections 2024-04-24T23:22:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:04.570+0000 7fba05053640 1 -- 172.21.15.102:0/774048293 wait complete. 2024-04-24T23:22:05.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:05 smithi102 ceph-mon[29216]: pgmap v278: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 36 MiB/s wr, 9.13k op/s 2024-04-24T23:22:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:05 smithi161 ceph-mon[32709]: pgmap v278: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 36 MiB/s wr, 9.13k op/s 2024-04-24T23:22:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:05 smithi183 ceph-mon[33137]: pgmap v278: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 36 MiB/s wr, 9.13k op/s 2024-04-24T23:22:07.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:07 smithi102 ceph-mon[29216]: pgmap v279: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 6.01k op/s 2024-04-24T23:22:07.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:07 smithi102 ceph-mon[29216]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:22:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:07 smithi161 ceph-mon[32709]: pgmap v279: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 6.01k op/s 2024-04-24T23:22:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:07 smithi161 ceph-mon[32709]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:22:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:07 smithi183 ceph-mon[33137]: pgmap v279: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 6.01k op/s 2024-04-24T23:22:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:07 smithi183 ceph-mon[33137]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:22:09.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:09 smithi102 ceph-mon[29216]: pgmap v281: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 27 MiB/s wr, 6.92k op/s 2024-04-24T23:22:09.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:09 smithi161 ceph-mon[32709]: pgmap v281: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 27 MiB/s wr, 6.92k op/s 2024-04-24T23:22:09.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:09 smithi183 ceph-mon[33137]: pgmap v281: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 27 MiB/s wr, 6.92k op/s 2024-04-24T23:22:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:10 smithi183 ceph-mon[33137]: pgmap v282: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 27 MiB/s wr, 6.92k op/s 2024-04-24T23:22:11.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:10 smithi183 ceph-mon[33137]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:22:11.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:10 smithi102 ceph-mon[29216]: pgmap v282: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 27 MiB/s wr, 6.92k op/s 2024-04-24T23:22:11.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:10 smithi102 ceph-mon[29216]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:22:11.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:10 smithi161 ceph-mon[32709]: pgmap v282: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 27 MiB/s wr, 6.92k op/s 2024-04-24T23:22:11.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:10 smithi161 ceph-mon[32709]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:22:13.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:12 smithi183 ceph-mon[33137]: pgmap v284: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2024-04-24T23:22:13.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:12 smithi102 ceph-mon[29216]: pgmap v284: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2024-04-24T23:22:13.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:12 smithi161 ceph-mon[32709]: pgmap v284: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2024-04-24T23:22:15.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:14 smithi183 ceph-mon[33137]: pgmap v285: 97 pgs: 97 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 35 MiB/s wr, 8.81k op/s 2024-04-24T23:22:15.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:14 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:14 smithi102 ceph-mon[29216]: pgmap v285: 97 pgs: 97 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 35 MiB/s wr, 8.81k op/s 2024-04-24T23:22:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:14 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:14 smithi161 ceph-mon[32709]: pgmap v285: 97 pgs: 97 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 35 MiB/s wr, 8.81k op/s 2024-04-24T23:22:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:14 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:16.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:15 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:16.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:15 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:16.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:15 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:22:16.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:15 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:16.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:15 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:16.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:15 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:22:16.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:15 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:16.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:15 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:16.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:15 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:22:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.716+0000 7f6f3b31b640 1 -- 172.21.15.102:0/1619047689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341090e0 msgr2=0x7f6f34115260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.716+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/1619047689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341090e0 0x7f6f34115260 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009ca0 tx=0x7f6f2c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.716+0000 7f6f3b31b640 1 -- 172.21.15.102:0/1619047689 shutdown_connections 2024-04-24T23:22:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.716+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/1619047689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341090e0 0x7f6f34115260 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.716+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/1619047689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f34108740 0x7f6f34108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.716+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/1619047689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3410e5c0 0x7f6f3410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.716+0000 7f6f3b31b640 1 -- 172.21.15.102:0/1619047689 >> 172.21.15.102:0/1619047689 conn(0x7f6f34073430 msgr2=0x7f6f3406d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.717+0000 7f6f3b31b640 1 -- 172.21.15.102:0/1619047689 shutdown_connections 2024-04-24T23:22:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.717+0000 7f6f3b31b640 1 -- 172.21.15.102:0/1619047689 wait complete. 2024-04-24T23:22:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.717+0000 7f6f3b31b640 1 Processor -- start 2024-04-24T23:22:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.718+0000 7f6f3b31b640 1 -- start start 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.718+0000 7f6f3b31b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f34108740 0x7f6f341a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.718+0000 7f6f3b31b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f341090e0 0x7f6f341a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.718+0000 7f6f3b31b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3410e5c0 0x7f6f341a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.718+0000 7f6f3b31b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341186c0 con 0x7f6f34108740 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.718+0000 7f6f388b6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f341090e0 0x7f6f341a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f390b7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f34108740 0x7f6f341a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f388b6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f341090e0 0x7f6f341a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36216/0 (socket says 172.21.15.102:36216) 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f388b6640 1 -- 172.21.15.102:0/3020933332 learned_addr learned my addr 172.21.15.102:0/3020933332 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34118540 con 0x7f6f3410e5c0 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f398b8640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3410e5c0 0x7f6f341a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34118840 con 0x7f6f341090e0 2024-04-24T23:22:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f398b8640 1 -- 172.21.15.102:0/3020933332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f341090e0 msgr2=0x7f6f341a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f398b8640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f341090e0 0x7f6f341a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f398b8640 1 -- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f34108740 msgr2=0x7f6f341a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f398b8640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f34108740 0x7f6f341a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f398b8640 1 -- 172.21.15.102:0/3020933332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f341a5500 con 0x7f6f3410e5c0 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f388b6640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f341090e0 0x7f6f341a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.719+0000 7f6f390b7640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f34108740 0x7f6f341a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.720+0000 7f6f398b8640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3410e5c0 0x7f6f341a4dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009ca0 tx=0x7f6f2c031680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.720+0000 7f6f2a7fc640 1 -- 172.21.15.102:0/3020933332 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c031e40 con 0x7f6f3410e5c0 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.720+0000 7f6f2a7fc640 1 -- 172.21.15.102:0/3020933332 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2c040070 con 0x7f6f3410e5c0 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.720+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f341a5790 con 0x7f6f3410e5c0 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.720+0000 7f6f2a7fc640 1 -- 172.21.15.102:0/3020933332 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0045f0 con 0x7f6f3410e5c0 2024-04-24T23:22:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.720+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341ad080 con 0x7f6f3410e5c0 2024-04-24T23:22:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.723+0000 7f6f3e02b740 1 Processor -- start 2024-04-24T23:22:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.723+0000 7f6f2a7fc640 1 -- 172.21.15.102:0/3020933332 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6f2c003970 con 0x7f6f3410e5c0 2024-04-24T23:22:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.723+0000 7f6f3e02b740 1 -- start start 2024-04-24T23:22:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.724+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 0x563e7bdc7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.724+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc7cd0 0x563e7bdd1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.724+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdd4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.724+0000 7f6f3e02b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563e7bd2bcb0 con 0x563e7bdc73b0 2024-04-24T23:22:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.724+0000 7f6f3e02b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563e7bcd45b0 con 0x563e7bdd23a0 2024-04-24T23:22:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.724+0000 7f6f3e02b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e7bd27c70 con 0x563e7bdc7cd0 2024-04-24T23:22:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.724+0000 7f6f388b6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdd4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.724+0000 7f6f388b6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdd4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:51832/0 (socket says 172.21.15.102:51832) 2024-04-24T23:22:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.724+0000 7f6f388b6640 1 -- 172.21.15.102:0/4279163590 learned_addr learned my addr 172.21.15.102:0/4279163590 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f388b6640 1 -- 172.21.15.102:0/4279163590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc7cd0 msgr2=0x563e7bdd1e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f388b6640 1 --2- 172.21.15.102:0/4279163590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc7cd0 0x563e7bdd1e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f388b6640 1 -- 172.21.15.102:0/4279163590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 msgr2=0x563e7bdc7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f388b6640 1 --2- 172.21.15.102:0/4279163590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 0x563e7bdc7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f388b6640 1 -- 172.21.15.102:0/4279163590 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e7ba40210 con 0x563e7bdd23a0 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f388b6640 1 --2- 172.21.15.102:0/4279163590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdd4790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400bec0 tx=0x7f6f24012ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=721a05f94c9623e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f2a7fc640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f200a2df0 0x7f6f200a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f1f7fe640 1 -- 172.21.15.102:0/4279163590 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24005220 con 0x563e7bdd23a0 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f1f7fe640 1 -- 172.21.15.102:0/4279163590 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2401b070 con 0x563e7bdd23a0 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f390b7640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f200a2df0 0x7f6f200a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f2a7fc640 1 -- 172.21.15.102:0/3020933332 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f6f2c0f0700 con 0x7f6f3410e5c0 2024-04-24T23:22:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.725+0000 7f6f1f7fe640 1 -- 172.21.15.102:0/4279163590 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f240109f0 con 0x563e7bdd23a0 2024-04-24T23:22:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.726+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4279163590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 msgr2=0x563e7bdd4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.726+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4279163590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdd4790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400bec0 tx=0x7f6f24012ae0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.726+0000 7f6f390b7640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f200a2df0 0x7f6f200a52b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x563e7bdc8970 tx=0x7f6f30009110 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.727+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4279163590 shutdown_connections 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.727+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4279163590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdd4790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.727+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4279163590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc7cd0 0x563e7bdd1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.727+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4279163590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 0x563e7bdc7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.727+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4279163590 >> 172.21.15.102:0/4279163590 conn(0x563e7bdc36d0 msgr2=0x563e7bdc3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.727+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4279163590 shutdown_connections 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.727+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4279163590 wait complete. 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.727+0000 7f6f3e02b740 1 -- start start 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.727+0000 7f6f3e02b740 1 Processor -- start 2024-04-24T23:22:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 0x563e7bdcf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc7cd0 0x563e7bdcf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f3e02b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563e7bdd5190 con 0x563e7bdc73b0 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f3e02b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563e7bce7b50 con 0x563e7bdc7cd0 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f3e02b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e7bdd5310 con 0x563e7bdd23a0 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f390b7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 0x563e7bdcf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f390b7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 0x563e7bdcf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58954/0 (socket says 172.21.15.102:58954) 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f3e02b740 10 client.0 _ll_register_callbacks cb 0x563e7bd22ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f390b7640 1 -- 172.21.15.102:0/4038104853 learned_addr learned my addr 172.21.15.102:0/4038104853 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f388b6640 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc7cd0 0x563e7bdcf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f398b8640 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f390b7640 1 -- 172.21.15.102:0/4038104853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 msgr2=0x563e7bdcfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f390b7640 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f390b7640 1 -- 172.21.15.102:0/4038104853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc7cd0 msgr2=0x563e7bdcf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f390b7640 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc7cd0 0x563e7bdcf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.728+0000 7f6f390b7640 1 -- 172.21.15.102:0/4038104853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e7bb0c620 con 0x563e7bdc73b0 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.729+0000 7f6f398b8640 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.729+0000 7f6f390b7640 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 0x563e7bdcf340 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6f30015a90 tx=0x7f6f300047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f34265934ce32 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.729+0000 7f6f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.729+0000 7f6f1cff9640 1 -- 172.21.15.102:0/4038104853 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30023020 con 0x563e7bdc73b0 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.729+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4038104853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f2401b070 con 0x563e7bdc73b0 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.729+0000 7f6f1cff9640 1 -- 172.21.15.102:0/4038104853 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f6f30013a10 con 0x563e7bdc73b0 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f1cff9640 1 -- 172.21.15.102:0/4038104853 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f3002e050 con 0x563e7bdc73b0 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f3e02b740 10 client.16140 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4038104853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e7bde9c50 con 0x563e7bdc73b0 2024-04-24T23:22:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f1cff9640 1 -- 172.21.15.102:0/4038104853 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2401b070 con 0x563e7bdc73b0 2024-04-24T23:22:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f1cff9640 1 -- 172.21.15.102:0/4038104853 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6f3001e030 con 0x563e7bdc73b0 2024-04-24T23:22:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f3e02b740 10 client.16140 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f3e02b740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f3e02b740 20 client.16140 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563e7bdeb580 0x563e7bdeda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f3e02b740 4 client.16140 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4038104853 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e7bd391f0 con 0x563e7bdeb580 2024-04-24T23:22:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.730+0000 7f6f388b6640 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563e7bdeb580 0x563e7bdeda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.731+0000 7f6f1cff9640 10 client.16140 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.731+0000 7f6f388b6640 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563e7bdeb580 0x563e7bdeda00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916fa25e2a5d69e5 server_cookie=7959656352bafdfb in_seq=0 out_seq=0 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.732+0000 7f6f1cff9640 1 -- 172.21.15.102:0/4038104853 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e7bd391f0 con 0x563e7bdeb580 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.732+0000 7f6f1cff9640 10 client.16140 handle_command_reply: tid=0 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.732+0000 7f6f3e02b740 1 client.16140 shutdown 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.732+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4038104853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 msgr2=0x563e7bdcf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.732+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 0x563e7bdcf340 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6f30015a90 tx=0x7f6f300047d0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4038104853 shutdown_connections 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc7cd0 0x563e7bdcf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563e7bdeb580 0x563e7bdeda00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/4038104853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc73b0 0x563e7bdcf340 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4038104853 >> 172.21.15.102:0/4038104853 conn(0x563e7bdc36d0 msgr2=0x563e7bdc94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f1cff9640 0 client.16140 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4038104853 shutdown_connections 2024-04-24T23:22:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f3e02b740 1 -- 172.21.15.102:0/4038104853 wait complete. 2024-04-24T23:22:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.733+0000 7f6f3e02b740 20 client.16140 trim_cache size 0 max 16384 2024-04-24T23:22:17.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:17 smithi183 ceph-mon[33137]: pgmap v286: 97 pgs: 97 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 29 MiB/s wr, 7.34k op/s 2024-04-24T23:22:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.764+0000 7f6f3e02b740 1 Processor -- start 2024-04-24T23:22:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.764+0000 7f6f3e02b740 1 -- start start 2024-04-24T23:22:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc73b0 0x563e7bdde050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdc9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f390b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc73b0 0x563e7bdde050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f390b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc73b0 0x563e7bdde050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36246/0 (socket says 172.21.15.102:36246) 2024-04-24T23:22:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f390b7640 1 -- 172.21.15.102:0/2521816807 learned_addr learned my addr 172.21.15.102:0/2521816807 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdf32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f388b6640 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdc9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f3e02b740 1 -- 172.21.15.102:0/2521816807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b60b0 con 0x563e7bdc7cd0 2024-04-24T23:22:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f3e02b740 1 -- 172.21.15.102:0/2521816807 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b3980 con 0x563e7bdd23a0 2024-04-24T23:22:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.765+0000 7f6f3e02b740 1 -- 172.21.15.102:0/2521816807 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b17f0 con 0x563e7bdc73b0 2024-04-24T23:22:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.766+0000 7f6f398b8640 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdf32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.766+0000 7f6f388b6640 1 -- 172.21.15.102:0/2521816807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc73b0 msgr2=0x563e7bdde050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.766+0000 7f6f388b6640 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc73b0 0x563e7bdde050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.766+0000 7f6f388b6640 1 -- 172.21.15.102:0/2521816807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 msgr2=0x563e7bdf32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.766+0000 7f6f388b6640 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdf32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.766+0000 7f6f388b6640 1 -- 172.21.15.102:0/2521816807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e7bdc1bb0 con 0x563e7bdc7cd0 2024-04-24T23:22:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.766+0000 7f6f388b6640 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdc9b10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x563e7bdceba0 tx=0x7f6f2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79f187c1d87e518 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.766+0000 7f6f1effd640 1 -- 172.21.15.102:0/2521816807 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24019040 con 0x563e7bdc7cd0 2024-04-24T23:22:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.767+0000 7f6f1effd640 1 -- 172.21.15.102:0/2521816807 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2401b050 con 0x563e7bdc7cd0 2024-04-24T23:22:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.767+0000 7f6f1effd640 1 -- 172.21.15.102:0/2521816807 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24010c80 con 0x563e7bdc7cd0 2024-04-24T23:22:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.767+0000 7f6f3e02b740 1 -- 172.21.15.102:0/2521816807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 msgr2=0x563e7bdc9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.767+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdc9b10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x563e7bdceba0 tx=0x7f6f2400be50 comp rx=0 tx=0).stop 2024-04-24T23:22:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.768+0000 7f6f3e02b740 1 -- 172.21.15.102:0/2521816807 shutdown_connections 2024-04-24T23:22:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.768+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdd23a0 0x563e7bdf32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.768+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdc9b10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.768+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/2521816807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdc73b0 0x563e7bdde050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.768+0000 7f6f3e02b740 1 -- 172.21.15.102:0/2521816807 >> 172.21.15.102:0/2521816807 conn(0x563e7bdc36d0 msgr2=0x563e7bdc2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.768+0000 7f6f3e02b740 1 -- 172.21.15.102:0/2521816807 shutdown_connections 2024-04-24T23:22:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.768+0000 7f6f3e02b740 1 -- 172.21.15.102:0/2521816807 wait complete. 2024-04-24T23:22:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.768+0000 7f6f3e02b740 1 -- start start 2024-04-24T23:22:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.768+0000 7f6f3e02b740 1 Processor -- start 2024-04-24T23:22:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc73b0 0x563e7bdcc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdcc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f3e02b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f3e02b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b1e40 con 0x563e7bdc7cd0 2024-04-24T23:22:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f3e02b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b32b0 con 0x563e7bdc73b0 2024-04-24T23:22:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f3e02b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341b2510 con 0x563e7bdd23a0 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f3e02b740 10 client.0 _ll_register_callbacks cb 0x7f6f30009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f398b8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f388b6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdcc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f398b8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36254/0 (socket says 172.21.15.102:36254) 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f398b8640 1 -- 172.21.15.102:0/1365810547 learned_addr learned my addr 172.21.15.102:0/1365810547 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f390b7640 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc73b0 0x563e7bdcc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f398b8640 1 -- 172.21.15.102:0/1365810547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc73b0 msgr2=0x563e7bdcc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f398b8640 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc73b0 0x563e7bdcc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f398b8640 1 -- 172.21.15.102:0/1365810547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 msgr2=0x563e7bdcc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f398b8640 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdcc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.770+0000 7f6f398b8640 1 -- 172.21.15.102:0/1365810547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e7be50b40 con 0x563e7bdd23a0 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.771+0000 7f6f388b6640 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdcc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.771+0000 7f6f398b8640 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcca90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c031040 tx=0x7f6f2c0bc5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b93f10aad3692a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.771+0000 7f6f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.771+0000 7f6f1cff9640 1 -- 172.21.15.102:0/1365810547 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0bcc90 con 0x563e7bdd23a0 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.771+0000 7f6f1cff9640 1 -- 172.21.15.102:0/1365810547 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f6f2c003c60 con 0x563e7bdd23a0 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.771+0000 7f6f3e02b740 1 -- 172.21.15.102:0/1365810547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f2401b050 con 0x563e7bdd23a0 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.771+0000 7f6f1cff9640 1 -- 172.21.15.102:0/1365810547 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c0c1040 con 0x563e7bdd23a0 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.771+0000 7f6f3e02b740 10 client.25474 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.771+0000 7f6f3e02b740 1 -- 172.21.15.102:0/1365810547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e7be51e90 con 0x563e7bdd23a0 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.772+0000 7f6f1cff9640 1 -- 172.21.15.102:0/1365810547 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2401b050 con 0x563e7bdd23a0 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.772+0000 7f6f1cff9640 1 -- 172.21.15.102:0/1365810547 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6f2c0db020 con 0x563e7bdd23a0 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.772+0000 7f6f3e02b740 10 client.25474 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.772+0000 7f6f3e02b740 10 client.25474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.772+0000 7f6f3e02b740 20 client.25474 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.772+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563e7be525e0 0x563e7be54a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.772+0000 7f6f3e02b740 4 client.25474 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:22:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.772+0000 7f6f3e02b740 1 -- 172.21.15.102:0/1365810547 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e7bd33540 con 0x563e7be525e0 2024-04-24T23:22:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.772+0000 7f6f390b7640 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563e7be525e0 0x563e7be54a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.773+0000 7f6f390b7640 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563e7be525e0 0x563e7be54a00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0b30121ed5bc4b server_cookie=c482a929aa587c8e in_seq=0 out_seq=0 2024-04-24T23:22:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.773+0000 7f6f1cff9640 10 client.25474 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.782+0000 7f6f1cff9640 1 -- 172.21.15.102:0/1365810547 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e7bd33540 con 0x563e7be525e0 2024-04-24T23:22:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.782+0000 7f6f1cff9640 10 client.25474 handle_command_reply: tid=0 2024-04-24T23:22:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.782+0000 7f6f3e02b740 1 client.25474 shutdown 2024-04-24T23:22:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.783+0000 7f6f3e02b740 1 -- 172.21.15.102:0/1365810547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 msgr2=0x563e7bdcca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.783+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcca90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c031040 tx=0x7f6f2c0bc5f0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.783+0000 7f6f3e02b740 1 -- 172.21.15.102:0/1365810547 shutdown_connections 2024-04-24T23:22:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.783+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e7bdd23a0 0x563e7bdcca90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.784+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x563e7be525e0 0x563e7be54a00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.784+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7bdc7cd0 0x563e7bdcc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.784+0000 7f6f3e02b740 1 --2- 172.21.15.102:0/1365810547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e7bdc73b0 0x563e7bdcc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.784+0000 7f6f3e02b740 1 -- 172.21.15.102:0/1365810547 >> 172.21.15.102:0/1365810547 conn(0x563e7bdc36d0 msgr2=0x563e7bdcf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.784+0000 7f6f1cff9640 0 client.25474 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.784+0000 7f6f3e02b740 1 -- 172.21.15.102:0/1365810547 shutdown_connections 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.784+0000 7f6f3e02b740 1 -- 172.21.15.102:0/1365810547 wait complete. 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.784+0000 7f6f3e02b740 20 client.25474 trim_cache size 0 max 16384 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:22:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:22:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.785+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f200a2df0 msgr2=0x7f6f200a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.785+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f200a2df0 0x7f6f200a52b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x563e7bdc8970 tx=0x7f6f30009110 comp rx=0 tx=0).stop 2024-04-24T23:22:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.785+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3410e5c0 msgr2=0x7f6f341a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.785+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3410e5c0 0x7f6f341a4dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009ca0 tx=0x7f6f2c031680 comp rx=0 tx=0).stop 2024-04-24T23:22:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.787+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 shutdown_connections 2024-04-24T23:22:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.787+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f3410e5c0 0x7f6f341a4dd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.787+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f341090e0 0x7f6f341a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.787+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6f200a2df0 0x7f6f200a52b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.787+0000 7f6f3b31b640 1 --2- 172.21.15.102:0/3020933332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f34108740 0x7f6f341a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.787+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 >> 172.21.15.102:0/3020933332 conn(0x7f6f34073430 msgr2=0x7f6f3406cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.787+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 shutdown_connections 2024-04-24T23:22:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:17.788+0000 7f6f3b31b640 1 -- 172.21.15.102:0/3020933332 wait complete. 2024-04-24T23:22:17.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:17 smithi102 ceph-mon[29216]: pgmap v286: 97 pgs: 97 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 29 MiB/s wr, 7.34k op/s 2024-04-24T23:22:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:17 smithi161 ceph-mon[32709]: pgmap v286: 97 pgs: 97 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 29 MiB/s wr, 7.34k op/s 2024-04-24T23:22:19.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:19 smithi183 ceph-mon[33137]: pgmap v287: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 7.07k op/s 2024-04-24T23:22:19.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:19 smithi102 ceph-mon[29216]: pgmap v287: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 7.07k op/s 2024-04-24T23:22:19.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:19 smithi161 ceph-mon[32709]: pgmap v287: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 7.07k op/s 2024-04-24T23:22:20.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:20 smithi102 ceph-mon[29216]: pgmap v288: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 7.07k op/s 2024-04-24T23:22:20.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:20 smithi161 ceph-mon[32709]: pgmap v288: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 7.07k op/s 2024-04-24T23:22:21.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:20 smithi183 ceph-mon[33137]: pgmap v288: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 28 MiB/s wr, 7.07k op/s 2024-04-24T23:22:23.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:22 smithi183 ceph-mon[33137]: pgmap v289: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 6.06k op/s 2024-04-24T23:22:23.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:22 smithi102 ceph-mon[29216]: pgmap v289: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 6.06k op/s 2024-04-24T23:22:23.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:22 smithi161 ceph-mon[32709]: pgmap v289: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 6.06k op/s 2024-04-24T23:22:25.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:24 smithi183 ceph-mon[33137]: pgmap v290: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 35 MiB/s wr, 8.95k op/s 2024-04-24T23:22:25.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:24 smithi102 ceph-mon[29216]: pgmap v290: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 35 MiB/s wr, 8.95k op/s 2024-04-24T23:22:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:24 smithi161 ceph-mon[32709]: pgmap v290: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 35 MiB/s wr, 8.95k op/s 2024-04-24T23:22:27.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:26 smithi183 ceph-mon[33137]: pgmap v291: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 5.99k op/s 2024-04-24T23:22:27.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:26 smithi102 ceph-mon[29216]: pgmap v291: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 5.99k op/s 2024-04-24T23:22:27.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:26 smithi161 ceph-mon[32709]: pgmap v291: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 5.99k op/s 2024-04-24T23:22:29.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:28 smithi183 ceph-mon[33137]: pgmap v292: 97 pgs: 97 active+clean; 14 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 34 MiB/s wr, 8.75k op/s 2024-04-24T23:22:29.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:28 smithi102 ceph-mon[29216]: pgmap v292: 97 pgs: 97 active+clean; 14 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 34 MiB/s wr, 8.75k op/s 2024-04-24T23:22:29.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:28 smithi161 ceph-mon[32709]: pgmap v292: 97 pgs: 97 active+clean; 14 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 34 MiB/s wr, 8.75k op/s 2024-04-24T23:22:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.933+0000 7f7aacd22640 1 -- 172.21.15.102:0/3851905559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa8100650 msgr2=0x7f7aa8100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.933+0000 7f7aacd22640 1 --2- 172.21.15.102:0/3851905559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa8100650 0x7f7aa8100ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009930 tx=0x7f7a9002a420 comp rx=0 tx=0).stop 2024-04-24T23:22:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.933+0000 7f7aacd22640 1 -- 172.21.15.102:0/3851905559 shutdown_connections 2024-04-24T23:22:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.933+0000 7f7aacd22640 1 --2- 172.21.15.102:0/3851905559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8101010 0x7f7aa810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.933+0000 7f7aacd22640 1 --2- 172.21.15.102:0/3851905559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa8100650 0x7f7aa8100ad0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009930 tx=0x7f7a9002a420 comp rx=0 tx=0).stop 2024-04-24T23:22:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.933+0000 7f7aacd22640 1 --2- 172.21.15.102:0/3851905559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aa8106660 0x7f7aa8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.933+0000 7f7aacd22640 1 -- 172.21.15.102:0/3851905559 >> 172.21.15.102:0/3851905559 conn(0x7f7aa8077850 msgr2=0x7f7aa8077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.934+0000 7f7aacd22640 1 -- 172.21.15.102:0/3851905559 shutdown_connections 2024-04-24T23:22:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.934+0000 7f7aacd22640 1 -- 172.21.15.102:0/3851905559 wait complete. 2024-04-24T23:22:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.936+0000 7f7aacd22640 1 Processor -- start 2024-04-24T23:22:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.936+0000 7f7aacd22640 1 -- start start 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7aacd22640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa8101010 0x7f7aa81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7aacd22640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8106660 0x7f7aa81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7aacd22640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aa819c970 0x7f7aa819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7aacd22640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8111a30 con 0x7f7aa8106660 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7aacd22640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81118b0 con 0x7f7aa819c970 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7aacd22640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8111bb0 con 0x7f7aa8101010 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8106660 0x7f7aa81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8106660 0x7f7aa81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46310/0 (socket says 172.21.15.102:46310) 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7a9ffff640 1 -- 172.21.15.102:0/1945127788 learned_addr learned my addr 172.21.15.102:0/1945127788 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7aa659c640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa8101010 0x7f7aa81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.937+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aa819c970 0x7f7aa819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7a9ffff640 1 -- 172.21.15.102:0/1945127788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa8101010 msgr2=0x7f7aa81980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa8101010 0x7f7aa81980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7a9ffff640 1 -- 172.21.15.102:0/1945127788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aa819c970 msgr2=0x7f7aa819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aa819c970 0x7f7aa819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7a9ffff640 1 -- 172.21.15.102:0/1945127788 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa819d460 con 0x7f7aa8106660 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aa819c970 0x7f7aa819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8106660 0x7f7aa81985f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009230 tx=0x7f7a900040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7a9f7fe640 1 -- 172.21.15.102:0/1945127788 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9003f030 con 0x7f7aa8106660 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7a9f7fe640 1 -- 172.21.15.102:0/1945127788 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a900044c0 con 0x7f7aa8106660 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7a9f7fe640 1 -- 172.21.15.102:0/1945127788 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90035070 con 0x7f7aa8106660 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.938+0000 7f7aacd22640 1 -- 172.21.15.102:0/1945127788 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa819d6f0 con 0x7f7aa8106660 2024-04-24T23:22:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.939+0000 7f7aacd22640 1 -- 172.21.15.102:0/1945127788 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa810df40 con 0x7f7aa8106660 2024-04-24T23:22:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.941+0000 7f7a9f7fe640 1 -- 172.21.15.102:0/1945127788 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f7a90042030 con 0x7f7aa8106660 2024-04-24T23:22:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.942+0000 7f7a9f7fe640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f7a940a2f00 0x7f7a940a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.943+0000 7f7a9f7fe640 1 -- 172.21.15.102:0/1945127788 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f7a900ef710 con 0x7f7aa8106660 2024-04-24T23:22:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.943+0000 7f7aa659c640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f7a940a2f00 0x7f7a940a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.944+0000 7f7aa659c640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f7a940a2f00 0x7f7a940a53c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005e00 tx=0x7f7a98005c50 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.944+0000 7f7aafa34740 1 Processor -- start 2024-04-24T23:22:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aafa34740 1 -- start start 2024-04-24T23:22:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb33688f90 0x55eb33689370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb3369c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb3369ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aafa34740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55eb335edcb0 con 0x55eb33689940 2024-04-24T23:22:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aafa34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb335965b0 con 0x55eb3369ca40 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aafa34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb335e9c70 con 0x55eb33688f90 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb3369c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aa6d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb3369ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aa659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb33688f90 0x55eb33689370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aa6d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb3369ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:44366/0 (socket says 172.21.15.102:44366) 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aa6d9d640 1 -- 172.21.15.102:0/4243600759 learned_addr learned my addr 172.21.15.102:0/4243600759 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aa6d9d640 1 -- 172.21.15.102:0/4243600759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb33688f90 msgr2=0x55eb33689370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.945+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/4243600759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb33688f90 0x55eb33689370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.946+0000 7f7aa6d9d640 1 -- 172.21.15.102:0/4243600759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 msgr2=0x55eb3369c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.946+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/4243600759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb3369c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.946+0000 7f7aa6d9d640 1 -- 172.21.15.102:0/4243600759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb33302210 con 0x55eb3369ca40 2024-04-24T23:22:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.946+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/4243600759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb3369c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.946+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/4243600759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb3369ee30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009fd0 tx=0x7f7aa00108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1028a9ab7c2533 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.946+0000 7f7a9cff9640 1 -- 172.21.15.102:0/4243600759 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0011760 con 0x55eb3369ca40 2024-04-24T23:22:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.946+0000 7f7a9cff9640 1 -- 172.21.15.102:0/4243600759 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa0011db0 con 0x55eb3369ca40 2024-04-24T23:22:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.946+0000 7f7a9cff9640 1 -- 172.21.15.102:0/4243600759 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000e3c0 con 0x55eb3369ca40 2024-04-24T23:22:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.948+0000 7f7aafa34740 1 -- 172.21.15.102:0/4243600759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 msgr2=0x55eb3369ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.948+0000 7f7aafa34740 1 --2- 172.21.15.102:0/4243600759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb3369ee30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009fd0 tx=0x7f7aa00108e0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.948+0000 7f7aafa34740 1 -- 172.21.15.102:0/4243600759 shutdown_connections 2024-04-24T23:22:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.948+0000 7f7aafa34740 1 --2- 172.21.15.102:0/4243600759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb3369ee30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.948+0000 7f7aafa34740 1 --2- 172.21.15.102:0/4243600759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb3369c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.948+0000 7f7aafa34740 1 --2- 172.21.15.102:0/4243600759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb33688f90 0x55eb33689370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.948+0000 7f7aafa34740 1 -- 172.21.15.102:0/4243600759 >> 172.21.15.102:0/4243600759 conn(0x55eb33685560 msgr2=0x55eb33685940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.948+0000 7f7aafa34740 1 -- 172.21.15.102:0/4243600759 shutdown_connections 2024-04-24T23:22:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.948+0000 7f7aafa34740 1 -- 172.21.15.102:0/4243600759 wait complete. 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.949+0000 7f7aafa34740 1 -- start start 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.949+0000 7f7aafa34740 1 Processor -- start 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.949+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb33688f90 0x55eb33699a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.949+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb33699fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.949+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb3369a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.949+0000 7f7aafa34740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55eb3369f800 con 0x55eb33689940 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.949+0000 7f7aafa34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb335a9b50 con 0x55eb33688f90 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.949+0000 7f7aafa34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb3369f980 con 0x55eb3369ca40 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.949+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb33699fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb33699fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46318/0 (socket says 172.21.15.102:46318) 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a9ffff640 1 -- 172.21.15.102:0/1560017969 learned_addr learned my addr 172.21.15.102:0/1560017969 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7aa659c640 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb33688f90 0x55eb33699a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb3369a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7aafa34740 10 client.0 _ll_register_callbacks cb 0x55eb335e4ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a9ffff640 1 -- 172.21.15.102:0/1560017969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 msgr2=0x55eb3369a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb3369a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a9ffff640 1 -- 172.21.15.102:0/1560017969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb33688f90 msgr2=0x55eb33699a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb33688f90 0x55eb33699a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a9ffff640 1 -- 172.21.15.102:0/1560017969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb333ce620 con 0x55eb33689940 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb33699fb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005940 tx=0x7f7a90003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65784c5fe41820e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a8dffb640 1 -- 172.21.15.102:0/1560017969 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900d2070 con 0x55eb33689940 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7a8dffb640 1 -- 172.21.15.102:0/1560017969 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f7a900c43d0 con 0x55eb33689940 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.950+0000 7f7aafa34740 1 -- 172.21.15.102:0/1560017969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa0011db0 con 0x55eb33689940 2024-04-24T23:22:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7a8dffb640 1 -- 172.21.15.102:0/1560017969 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a90049040 con 0x55eb33689940 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7aafa34740 10 client.16158 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7aafa34740 1 -- 172.21.15.102:0/1560017969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb336ab8c0 con 0x55eb33689940 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7a8dffb640 1 -- 172.21.15.102:0/1560017969 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa0011db0 con 0x55eb33689940 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7a8dffb640 1 -- 172.21.15.102:0/1560017969 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7a900da020 con 0x55eb33689940 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7aafa34740 10 client.16158 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7aafa34740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7aafa34740 20 client.16158 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7aafa34740 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb336acc80 0x55eb336af110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7aafa34740 4 client.16158 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.951+0000 7f7aafa34740 1 -- 172.21.15.102:0/1560017969 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb335fb1f0 con 0x55eb336acc80 2024-04-24T23:22:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.952+0000 7f7aa659c640 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb336acc80 0x55eb336af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.953+0000 7f7aa659c640 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb336acc80 0x55eb336af110 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3fc2ec8930c2f0 server_cookie=112621787826a993 in_seq=0 out_seq=0 2024-04-24T23:22:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.953+0000 7f7a8dffb640 10 client.16158 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.954+0000 7f7a8dffb640 1 -- 172.21.15.102:0/1560017969 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb335fb1f0 con 0x55eb336acc80 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.954+0000 7f7a8dffb640 10 client.16158 handle_command_reply: tid=0 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7aafa34740 1 client.16158 shutdown 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7aafa34740 1 -- 172.21.15.102:0/1560017969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 msgr2=0x55eb33699fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7aafa34740 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb33699fb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005940 tx=0x7f7a90003e40 comp rx=0 tx=0).stop 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7aafa34740 1 -- 172.21.15.102:0/1560017969 shutdown_connections 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7aafa34740 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb3369a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7aafa34740 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb336acc80 0x55eb336af110 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7aafa34740 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33689940 0x55eb33699fb0 secure :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005940 tx=0x7f7a90003e40 comp rx=0 tx=0).stop 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7aafa34740 1 --2- 172.21.15.102:0/1560017969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb33688f90 0x55eb33699a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7aafa34740 1 -- 172.21.15.102:0/1560017969 >> 172.21.15.102:0/1560017969 conn(0x55eb33685560 msgr2=0x55eb3368b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.955+0000 7f7a8dffb640 0 client.16158 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.956+0000 7f7aafa34740 1 -- 172.21.15.102:0/1560017969 shutdown_connections 2024-04-24T23:22:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.957+0000 7f7aafa34740 1 -- 172.21.15.102:0/1560017969 wait complete. 2024-04-24T23:22:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.957+0000 7f7aafa34740 20 client.16158 trim_cache size 0 max 16384 2024-04-24T23:22:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.986+0000 7f7aafa34740 1 Processor -- start 2024-04-24T23:22:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.986+0000 7f7aafa34740 1 -- start start 2024-04-24T23:22:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.986+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 0x55eb3369c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.986+0000 7f7aa659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 0x55eb3369c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.986+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb336a8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.986+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb336b4a20 0x55eb336b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.986+0000 7f7aa659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 0x55eb3369c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46326/0 (socket says 172.21.15.102:46326) 2024-04-24T23:22:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.986+0000 7f7aafa34740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81ae140 con 0x55eb33688f90 2024-04-24T23:22:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb336a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aafa34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81aba10 con 0x55eb3369ca40 2024-04-24T23:22:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aa659c640 1 -- 172.21.15.102:0/3307625484 learned_addr learned my addr 172.21.15.102:0/3307625484 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/3307625484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb336b4a20 0x55eb336b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aafa34740 1 -- 172.21.15.102:0/3307625484 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81a9880 con 0x55eb336b4a20 2024-04-24T23:22:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aa6d9d640 1 -- 172.21.15.102:0/3307625484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 msgr2=0x55eb336a8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/3307625484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb336a8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aa6d9d640 1 -- 172.21.15.102:0/3307625484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 msgr2=0x55eb3369c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/3307625484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 0x55eb3369c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aa6d9d640 1 -- 172.21.15.102:0/3307625484 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3369f160 con 0x55eb336b4a20 2024-04-24T23:22:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/3307625484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb336b4a20 0x55eb336b4ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000f80 tx=0x7f7aa0011f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6836e367b63d658f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.987+0000 7f7a8ffff640 1 -- 172.21.15.102:0/3307625484 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa002a070 con 0x55eb336b4a20 2024-04-24T23:22:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.988+0000 7f7a8ffff640 1 -- 172.21.15.102:0/3307625484 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa00049c0 con 0x55eb336b4a20 2024-04-24T23:22:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.988+0000 7f7a8ffff640 1 -- 172.21.15.102:0/3307625484 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0011a60 con 0x55eb336b4a20 2024-04-24T23:22:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.989+0000 7f7aafa34740 1 -- 172.21.15.102:0/3307625484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb336b4a20 msgr2=0x55eb336b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.989+0000 7f7aafa34740 1 --2- 172.21.15.102:0/3307625484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb336b4a20 0x55eb336b4ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000f80 tx=0x7f7aa0011f60 comp rx=0 tx=0).stop 2024-04-24T23:22:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.989+0000 7f7aafa34740 1 -- 172.21.15.102:0/3307625484 shutdown_connections 2024-04-24T23:22:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.989+0000 7f7aafa34740 1 --2- 172.21.15.102:0/3307625484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb336b4a20 0x55eb336b4ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.989+0000 7f7aafa34740 1 --2- 172.21.15.102:0/3307625484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb3369ca40 0x55eb336a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.989+0000 7f7aafa34740 1 --2- 172.21.15.102:0/3307625484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 0x55eb3369c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.989+0000 7f7aafa34740 1 -- 172.21.15.102:0/3307625484 >> 172.21.15.102:0/3307625484 conn(0x55eb33685560 msgr2=0x55eb33685940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.990+0000 7f7aafa34740 1 -- 172.21.15.102:0/3307625484 shutdown_connections 2024-04-24T23:22:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.990+0000 7f7aafa34740 1 -- 172.21.15.102:0/3307625484 wait complete. 2024-04-24T23:22:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.990+0000 7f7aafa34740 1 -- start start 2024-04-24T23:22:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.990+0000 7f7aafa34740 1 Processor -- start 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 0x55eb336a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb336a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7aafa34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb336b4a20 0x55eb336a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7aafa34740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81a9ed0 con 0x55eb33688f90 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7aafa34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81ab340 con 0x55eb336b4a20 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7aafa34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81aa5a0 con 0x55eb3369ca40 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7aafa34740 10 client.0 _ll_register_callbacks cb 0x7f7a900bb730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb336a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7aa659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 0x55eb336a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb336a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:47948/0 (socket says 172.21.15.102:47948) 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7a9ffff640 1 -- 172.21.15.102:0/2061258349 learned_addr learned my addr 172.21.15.102:0/2061258349 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7aa6d9d640 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb336b4a20 0x55eb336a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7a9ffff640 1 -- 172.21.15.102:0/2061258349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb336b4a20 msgr2=0x55eb336a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb336b4a20 0x55eb336a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7a9ffff640 1 -- 172.21.15.102:0/2061258349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 msgr2=0x55eb336a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 0x55eb336a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.991+0000 7f7a9ffff640 1 -- 172.21.15.102:0/2061258349 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb33723370 con 0x55eb3369ca40 2024-04-24T23:22:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.992+0000 7f7a9ffff640 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb336a0cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009900 tx=0x7f7a90005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bca70169fe089cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.992+0000 7f7a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.992+0000 7f7a8dffb640 1 -- 172.21.15.102:0/2061258349 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900d2040 con 0x55eb3369ca40 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.992+0000 7f7a8dffb640 1 -- 172.21.15.102:0/2061258349 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f7a900c43d0 con 0x55eb3369ca40 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.992+0000 7f7aafa34740 1 -- 172.21.15.102:0/2061258349 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa00049c0 con 0x55eb3369ca40 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.993+0000 7f7a8dffb640 1 -- 172.21.15.102:0/2061258349 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a900b7e90 con 0x55eb3369ca40 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.993+0000 7f7aafa34740 10 client.25492 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.993+0000 7f7aafa34740 1 -- 172.21.15.102:0/2061258349 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb33723370 con 0x55eb3369ca40 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.993+0000 7f7a8dffb640 1 -- 172.21.15.102:0/2061258349 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa00049c0 con 0x55eb3369ca40 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.993+0000 7f7a8dffb640 1 -- 172.21.15.102:0/2061258349 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7a900c4ab0 con 0x55eb3369ca40 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.994+0000 7f7aafa34740 10 client.25492 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.994+0000 7f7aafa34740 10 client.25492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.994+0000 7f7aafa34740 20 client.25492 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.994+0000 7f7aafa34740 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb33724a40 0x55eb33726e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.994+0000 7f7aafa34740 4 client.25492 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:22:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.994+0000 7f7aafa34740 1 -- 172.21.15.102:0/2061258349 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb335f5540 con 0x55eb33724a40 2024-04-24T23:22:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.994+0000 7f7aa659c640 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb33724a40 0x55eb33726e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.994+0000 7f7a8dffb640 10 client.25492 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:30.994+0000 7f7aa659c640 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb33724a40 0x55eb33726e60 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8696d9dc1b87c8 server_cookie=4243e055e36be3c8 in_seq=0 out_seq=0 2024-04-24T23:22:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.004+0000 7f7a8dffb640 1 -- 172.21.15.102:0/2061258349 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb335f5540 con 0x55eb33724a40 2024-04-24T23:22:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.004+0000 7f7a8dffb640 10 client.25492 handle_command_reply: tid=0 2024-04-24T23:22:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.004+0000 7f7aafa34740 1 client.25492 shutdown 2024-04-24T23:22:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.004+0000 7f7aafa34740 1 -- 172.21.15.102:0/2061258349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 msgr2=0x55eb336a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.004+0000 7f7aafa34740 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb336a0cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009900 tx=0x7f7a90005d50 comp rx=0 tx=0).stop 2024-04-24T23:22:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7aafa34740 1 -- 172.21.15.102:0/2061258349 shutdown_connections 2024-04-24T23:22:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7aafa34740 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb336b4a20 0x55eb336a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7aafa34740 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb3369ca40 0x55eb336a0cd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7aafa34740 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55eb33724a40 0x55eb33726e60 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7aafa34740 1 --2- 172.21.15.102:0/2061258349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eb33688f90 0x55eb336a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7aafa34740 1 -- 172.21.15.102:0/2061258349 >> 172.21.15.102:0/2061258349 conn(0x55eb33685560 msgr2=0x55eb33699a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7a8dffb640 0 client.25492 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7aafa34740 1 -- 172.21.15.102:0/2061258349 shutdown_connections 2024-04-24T23:22:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7aafa34740 1 -- 172.21.15.102:0/2061258349 wait complete. 2024-04-24T23:22:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.005+0000 7f7aafa34740 20 client.25492 trim_cache size 0 max 16384 2024-04-24T23:22:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:22:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:22:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:22:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:22:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.007+0000 7f7aacd22640 1 -- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f7a940a2f00 msgr2=0x7f7a940a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.007+0000 7f7aacd22640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f7a940a2f00 0x7f7a940a53c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005e00 tx=0x7f7a98005c50 comp rx=0 tx=0).stop 2024-04-24T23:22:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.007+0000 7f7aacd22640 1 -- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8106660 msgr2=0x7f7aa81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.007+0000 7f7aacd22640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8106660 0x7f7aa81985f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009230 tx=0x7f7a900040c0 comp rx=0 tx=0).stop 2024-04-24T23:22:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.007+0000 7f7aacd22640 1 -- 172.21.15.102:0/1945127788 shutdown_connections 2024-04-24T23:22:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.007+0000 7f7aacd22640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aa819c970 0x7f7aa819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.007+0000 7f7aacd22640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f7a940a2f00 0x7f7a940a53c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.007+0000 7f7aacd22640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8106660 0x7f7aa81985f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.008+0000 7f7aacd22640 1 --2- 172.21.15.102:0/1945127788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa8101010 0x7f7aa81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.008+0000 7f7aacd22640 1 -- 172.21.15.102:0/1945127788 >> 172.21.15.102:0/1945127788 conn(0x7f7aa8077850 msgr2=0x7f7aa8103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.008+0000 7f7aacd22640 1 -- 172.21.15.102:0/1945127788 shutdown_connections 2024-04-24T23:22:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:31.008+0000 7f7aacd22640 1 -- 172.21.15.102:0/1945127788 wait complete. 2024-04-24T23:22:31.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:30 smithi183 ceph-mon[33137]: pgmap v293: 97 pgs: 97 active+clean; 14 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 24 MiB/s wr, 6.15k op/s 2024-04-24T23:22:31.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:30 smithi102 ceph-mon[29216]: pgmap v293: 97 pgs: 97 active+clean; 14 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 24 MiB/s wr, 6.15k op/s 2024-04-24T23:22:31.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:30 smithi161 ceph-mon[32709]: pgmap v293: 97 pgs: 97 active+clean; 14 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 24 MiB/s wr, 6.15k op/s 2024-04-24T23:22:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:32 smithi183 ceph-mon[33137]: pgmap v294: 97 pgs: 97 active+clean; 14 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 24 MiB/s wr, 6.15k op/s 2024-04-24T23:22:33.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:32 smithi102 ceph-mon[29216]: pgmap v294: 97 pgs: 97 active+clean; 14 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 24 MiB/s wr, 6.15k op/s 2024-04-24T23:22:33.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:32 smithi161 ceph-mon[32709]: pgmap v294: 97 pgs: 97 active+clean; 14 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 24 MiB/s wr, 6.15k op/s 2024-04-24T23:22:35.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:34 smithi183 ceph-mon[33137]: pgmap v295: 97 pgs: 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 36 MiB/s wr, 9.18k op/s 2024-04-24T23:22:35.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:34 smithi102 ceph-mon[29216]: pgmap v295: 97 pgs: 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 36 MiB/s wr, 9.18k op/s 2024-04-24T23:22:35.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:34 smithi161 ceph-mon[32709]: pgmap v295: 97 pgs: 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 36 MiB/s wr, 9.18k op/s 2024-04-24T23:22:37.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:36 smithi183 ceph-mon[33137]: pgmap v296: 97 pgs: 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 24 MiB/s wr, 6.12k op/s 2024-04-24T23:22:37.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:36 smithi102 ceph-mon[29216]: pgmap v296: 97 pgs: 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 24 MiB/s wr, 6.12k op/s 2024-04-24T23:22:37.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:36 smithi161 ceph-mon[32709]: pgmap v296: 97 pgs: 97 active+clean; 14 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 24 MiB/s wr, 6.12k op/s 2024-04-24T23:22:39.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:38 smithi183 ceph-mon[33137]: pgmap v297: 97 pgs: 97 active+clean; 14 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 35 MiB/s wr, 8.87k op/s 2024-04-24T23:22:39.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:38 smithi102 ceph-mon[29216]: pgmap v297: 97 pgs: 97 active+clean; 14 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 35 MiB/s wr, 8.87k op/s 2024-04-24T23:22:39.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:38 smithi161 ceph-mon[32709]: pgmap v297: 97 pgs: 97 active+clean; 14 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 35 MiB/s wr, 8.87k op/s 2024-04-24T23:22:41.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:40 smithi183 ceph-mon[33137]: pgmap v298: 97 pgs: 97 active+clean; 14 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 25 MiB/s wr, 6.45k op/s 2024-04-24T23:22:41.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:40 smithi102 ceph-mon[29216]: pgmap v298: 97 pgs: 97 active+clean; 14 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 25 MiB/s wr, 6.45k op/s 2024-04-24T23:22:41.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:40 smithi161 ceph-mon[32709]: pgmap v298: 97 pgs: 97 active+clean; 14 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 25 MiB/s wr, 6.45k op/s 2024-04-24T23:22:43.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:42 smithi183 ceph-mon[33137]: pgmap v299: 97 pgs: 97 active+clean; 14 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 24 MiB/s wr, 6.13k op/s 2024-04-24T23:22:43.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:42 smithi102 ceph-mon[29216]: pgmap v299: 97 pgs: 97 active+clean; 14 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 24 MiB/s wr, 6.13k op/s 2024-04-24T23:22:43.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:42 smithi161 ceph-mon[32709]: pgmap v299: 97 pgs: 97 active+clean; 14 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 24 MiB/s wr, 6.13k op/s 2024-04-24T23:22:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.158+0000 7f8b0c970640 1 -- 172.21.15.102:0/2396769964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b08106500 msgr2=0x7f8b081068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.158+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2396769964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b08106500 0x7f8b081068e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009900 tx=0x7f8af802a3f0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.158+0000 7f8b0c970640 1 -- 172.21.15.102:0/2396769964 shutdown_connections 2024-04-24T23:22:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.158+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2396769964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08100e20 0x7f8b0810d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.158+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2396769964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b08100460 0x7f8b081008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.158+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2396769964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b08106500 0x7f8b081068e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.158+0000 7f8b0c970640 1 -- 172.21.15.102:0/2396769964 >> 172.21.15.102:0/2396769964 conn(0x7f8b080fbf90 msgr2=0x7f8b080fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.158+0000 7f8b0c970640 1 -- 172.21.15.102:0/2396769964 shutdown_connections 2024-04-24T23:22:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.158+0000 7f8b0c970640 1 -- 172.21.15.102:0/2396769964 wait complete. 2024-04-24T23:22:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.159+0000 7f8b0c970640 1 Processor -- start 2024-04-24T23:22:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.159+0000 7f8b0c970640 1 -- start start 2024-04-24T23:22:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.159+0000 7f8b0c970640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b08100460 0x7f8b081095a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.159+0000 7f8b0c970640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b08100e20 0x7f8b08109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.159+0000 7f8b0c970640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08106500 0x7f8b0810a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.159+0000 7f8b0c970640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810f630 con 0x7f8b08106500 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.159+0000 7f8b0c970640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810f4b0 con 0x7f8b08100e20 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.159+0000 7f8b0c970640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810f7b0 con 0x7f8b08100460 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8b0659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b08100460 0x7f8b081095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8afffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b08100e20 0x7f8b08109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8b0659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b08100460 0x7f8b081095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36314/0 (socket says 172.21.15.102:36314) 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8b0659c640 1 -- 172.21.15.102:0/2929999956 learned_addr learned my addr 172.21.15.102:0/2929999956 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8b06d9d640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08106500 0x7f8b0810a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8afffff640 1 -- 172.21.15.102:0/2929999956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b08100460 msgr2=0x7f8b081095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8afffff640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b08100460 0x7f8b081095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8afffff640 1 -- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08106500 msgr2=0x7f8b0810a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8afffff640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08106500 0x7f8b0810a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8afffff640 1 -- 172.21.15.102:0/2929999956 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b081aa670 con 0x7f8b08100e20 2024-04-24T23:22:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8b0659c640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b08100460 0x7f8b081095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.160+0000 7f8b06d9d640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08106500 0x7f8b0810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.161+0000 7f8afffff640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b08100e20 0x7f8b08109ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8af000d6d0 tx=0x7f8af000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.161+0000 7f8aff7fe640 1 -- 172.21.15.102:0/2929999956 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0017070 con 0x7f8b08100e20 2024-04-24T23:22:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.161+0000 7f8b0c970640 1 -- 172.21.15.102:0/2929999956 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b081aa960 con 0x7f8b08100e20 2024-04-24T23:22:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.161+0000 7f8b0c970640 1 -- 172.21.15.102:0/2929999956 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b081aaf00 con 0x7f8b08100e20 2024-04-24T23:22:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.162+0000 7f8aff7fe640 1 -- 172.21.15.102:0/2929999956 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8af0009820 con 0x7f8b08100e20 2024-04-24T23:22:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.162+0000 7f8aff7fe640 1 -- 172.21.15.102:0/2929999956 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0004c60 con 0x7f8b08100e20 2024-04-24T23:22:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.165+0000 7f8b0f67d740 1 Processor -- start 2024-04-24T23:22:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.165+0000 7f8b0f67d740 1 -- start start 2024-04-24T23:22:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.165+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 0x55e58c2cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.165+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.165+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d83a0 0x55e58c2da790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.165+0000 7f8b0f67d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e58c231cb0 con 0x55e58c2d83a0 2024-04-24T23:22:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.165+0000 7f8b0f67d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e58c1da5b0 con 0x55e58c2cdcd0 2024-04-24T23:22:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.165+0000 7f8b0f67d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e58c22dc70 con 0x55e58c2cd3b0 2024-04-24T23:22:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.166+0000 7f8afffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d83a0 0x55e58c2da790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.166+0000 7f8afffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d83a0 0x55e58c2da790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39372/0 (socket says 172.21.15.102:39372) 2024-04-24T23:22:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.166+0000 7f8afffff640 1 -- 172.21.15.102:0/53772414 learned_addr learned my addr 172.21.15.102:0/53772414 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.167+0000 7f8afffff640 1 -- 172.21.15.102:0/53772414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 msgr2=0x55e58c2cd790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:22:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.167+0000 7f8afffff640 1 --2- 172.21.15.102:0/53772414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 0x55e58c2cd790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.167+0000 7f8afffff640 1 -- 172.21.15.102:0/53772414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 msgr2=0x55e58c2d7e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:22:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.167+0000 7f8afffff640 1 --2- 172.21.15.102:0/53772414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d7e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.167+0000 7f8afffff640 1 -- 172.21.15.102:0/53772414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e58bf46210 con 0x55e58c2d83a0 2024-04-24T23:22:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.167+0000 7f8afffff640 1 --2- 172.21.15.102:0/53772414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d83a0 0x55e58c2da790 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8af00cbfb0 tx=0x7f8af00d2f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc87074837cd069 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.168+0000 7f8afcff9640 1 -- 172.21.15.102:0/53772414 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af00e2070 con 0x55e58c2d83a0 2024-04-24T23:22:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.168+0000 7f8aff7fe640 1 -- 172.21.15.102:0/2929999956 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8af0005220 con 0x7f8b08100e20 2024-04-24T23:22:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.168+0000 7f8afcff9640 1 -- 172.21.15.102:0/53772414 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8af00d7d40 con 0x55e58c2d83a0 2024-04-24T23:22:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.168+0000 7f8afcff9640 1 -- 172.21.15.102:0/53772414 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af00d9040 con 0x55e58c2d83a0 2024-04-24T23:22:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8aff7fe640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8af40a2fa0 0x7f8af40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b0f67d740 1 -- 172.21.15.102:0/53772414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d83a0 msgr2=0x55e58c2da790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/53772414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d83a0 0x55e58c2da790 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8af00cbfb0 tx=0x7f8af00d2f60 comp rx=0 tx=0).stop 2024-04-24T23:22:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b0f67d740 1 -- 172.21.15.102:0/53772414 shutdown_connections 2024-04-24T23:22:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/53772414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d83a0 0x55e58c2da790 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/53772414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/53772414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 0x55e58c2cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b0f67d740 1 -- 172.21.15.102:0/53772414 >> 172.21.15.102:0/53772414 conn(0x55e58c2c96d0 msgr2=0x55e58c2c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b06d9d640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8af40a2fa0 0x7f8af40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8aff7fe640 1 -- 172.21.15.102:0/2929999956 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f8af0004a00 con 0x7f8b08100e20 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b0f67d740 1 -- 172.21.15.102:0/53772414 shutdown_connections 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.169+0000 7f8b0f67d740 1 -- 172.21.15.102:0/53772414 wait complete. 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.170+0000 7f8b06d9d640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8af40a2fa0 0x7f8af40a5460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000a710 tx=0x7f8b0000d040 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.170+0000 7f8b0f67d740 1 -- start start 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.170+0000 7f8b0f67d740 1 Processor -- start 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.170+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cd3b0 0x55e58c2d54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.170+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.170+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2d5f20 0x55e58c2f5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.170+0000 7f8b0f67d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e58c2ec190 con 0x55e58c2cd3b0 2024-04-24T23:22:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.170+0000 7f8b0f67d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e58c1edb50 con 0x55e58c2d5f20 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.170+0000 7f8b0f67d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e58c2ec310 con 0x55e58c2cdcd0 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cd3b0 0x55e58c2d54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cd3b0 0x55e58c2d54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39386/0 (socket says 172.21.15.102:39386) 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0659c640 1 -- 172.21.15.102:0/2422557157 learned_addr learned my addr 172.21.15.102:0/2422557157 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0f67d740 10 client.0 _ll_register_callbacks cb 0x55e58c228ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b06d9d640 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2d5f20 0x55e58c2f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8afffff640 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0659c640 1 -- 172.21.15.102:0/2422557157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cdcd0 msgr2=0x55e58c2d59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0659c640 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0659c640 1 -- 172.21.15.102:0/2422557157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2d5f20 msgr2=0x55e58c2f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0659c640 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2d5f20 0x55e58c2f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0659c640 1 -- 172.21.15.102:0/2422557157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e58c012620 con 0x55e58c2cd3b0 2024-04-24T23:22:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b06d9d640 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2d5f20 0x55e58c2f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8afffff640 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8b0659c640 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cd3b0 0x55e58c2d54a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009a20 tx=0x7f8af80317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd39ced347c0512 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8aedffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8aedffb640 1 -- 172.21.15.102:0/2422557157 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8040070 con 0x55e58c2cd3b0 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.171+0000 7f8aedffb640 1 -- 172.21.15.102:0/2422557157 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f8af8004450 con 0x55e58c2cd3b0 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.172+0000 7f8b0f67d740 1 -- 172.21.15.102:0/2422557157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af00d7d40 con 0x55e58c2cd3b0 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.172+0000 7f8aedffb640 1 -- 172.21.15.102:0/2422557157 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8af8035040 con 0x55e58c2cd3b0 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.172+0000 7f8b0f67d740 10 client.16173 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.172+0000 7f8b0f67d740 1 -- 172.21.15.102:0/2422557157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e58c2f89e0 con 0x55e58c2cd3b0 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.172+0000 7f8aedffb640 1 -- 172.21.15.102:0/2422557157 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8af00d7d40 con 0x55e58c2cd3b0 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.172+0000 7f8aedffb640 1 -- 172.21.15.102:0/2422557157 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8af8004c90 con 0x55e58c2cd3b0 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.172+0000 7f8b0f67d740 10 client.16173 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.172+0000 7f8b0f67d740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.172+0000 7f8b0f67d740 20 client.16173 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.173+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55e58c2f9ea0 0x55e58c2fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.173+0000 7f8b0f67d740 4 client.16173 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.173+0000 7f8b0f67d740 1 -- 172.21.15.102:0/2422557157 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e58c23f1f0 con 0x55e58c2f9ea0 2024-04-24T23:22:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.173+0000 7f8afffff640 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55e58c2f9ea0 0x55e58c2fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.174+0000 7f8aedffb640 10 client.16173 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.174+0000 7f8afffff640 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55e58c2f9ea0 0x55e58c2fc2f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74b5cf48c5dcbf9 server_cookie=26e272f29e47fcc8 in_seq=0 out_seq=0 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.174+0000 7f8aedffb640 1 -- 172.21.15.102:0/2422557157 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e58c23f1f0 con 0x55e58c2f9ea0 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.174+0000 7f8aedffb640 10 client.16173 handle_command_reply: tid=0 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.174+0000 7f8b0f67d740 1 client.16173 shutdown 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 -- 172.21.15.102:0/2422557157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cd3b0 msgr2=0x55e58c2d54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cd3b0 0x55e58c2d54a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009a20 tx=0x7f8af80317a0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 -- 172.21.15.102:0/2422557157 shutdown_connections 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2d5f20 0x55e58c2f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55e58c2f9ea0 0x55e58c2fc2f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/2422557157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cd3b0 0x55e58c2d54a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 -- 172.21.15.102:0/2422557157 >> 172.21.15.102:0/2422557157 conn(0x55e58c2c96d0 msgr2=0x55e58c2cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8aedffb640 0 client.16173 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 -- 172.21.15.102:0/2422557157 shutdown_connections 2024-04-24T23:22:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 1 -- 172.21.15.102:0/2422557157 wait complete. 2024-04-24T23:22:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.175+0000 7f8b0f67d740 20 client.16173 trim_cache size 0 max 16384 2024-04-24T23:22:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.203+0000 7f8b0f67d740 1 Processor -- start 2024-04-24T23:22:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0f67d740 1 -- start start 2024-04-24T23:22:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cd3b0 0x55e58c2cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cdcd0 0x55e58c301c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2d5f20 0x55e58c302140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0f67d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae8011120 con 0x55e58c2cdcd0 2024-04-24T23:22:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0f67d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae800e9f0 con 0x55e58c2cd3b0 2024-04-24T23:22:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0f67d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae800c860 con 0x55e58c2d5f20 2024-04-24T23:22:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cd3b0 0x55e58c2cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cd3b0 0x55e58c2cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:44542/0 (socket says 172.21.15.102:44542) 2024-04-24T23:22:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8afffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cdcd0 0x55e58c301c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.204+0000 7f8b0659c640 1 -- 172.21.15.102:0/6554916 learned_addr learned my addr 172.21.15.102:0/6554916 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.205+0000 7f8b06d9d640 1 --2- 172.21.15.102:0/6554916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2d5f20 0x55e58c302140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.205+0000 7f8afffff640 1 -- 172.21.15.102:0/6554916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2d5f20 msgr2=0x55e58c302140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.205+0000 7f8afffff640 1 --2- 172.21.15.102:0/6554916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2d5f20 0x55e58c302140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.205+0000 7f8afffff640 1 -- 172.21.15.102:0/6554916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cd3b0 msgr2=0x55e58c2cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.205+0000 7f8afffff640 1 --2- 172.21.15.102:0/6554916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cd3b0 0x55e58c2cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.205+0000 7f8afffff640 1 -- 172.21.15.102:0/6554916 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e58c2cc9c0 con 0x55e58c2cdcd0 2024-04-24T23:22:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.205+0000 7f8afffff640 1 --2- 172.21.15.102:0/6554916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cdcd0 0x55e58c301c00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55e58c2ceb40 tx=0x7f8af0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0f41d76b1dc102 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.206+0000 7f8aeffff640 1 -- 172.21.15.102:0/6554916 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af009d070 con 0x55e58c2cdcd0 2024-04-24T23:22:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.206+0000 7f8aeffff640 1 -- 172.21.15.102:0/6554916 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8af00d6070 con 0x55e58c2cdcd0 2024-04-24T23:22:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.206+0000 7f8aeffff640 1 -- 172.21.15.102:0/6554916 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af00d9040 con 0x55e58c2cdcd0 2024-04-24T23:22:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 -- 172.21.15.102:0/6554916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cdcd0 msgr2=0x55e58c301c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/6554916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cdcd0 0x55e58c301c00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55e58c2ceb40 tx=0x7f8af0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 -- 172.21.15.102:0/6554916 shutdown_connections 2024-04-24T23:22:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/6554916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2d5f20 0x55e58c302140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/6554916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2cdcd0 0x55e58c301c00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/6554916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cd3b0 0x55e58c2cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 -- 172.21.15.102:0/6554916 >> 172.21.15.102:0/6554916 conn(0x55e58c2c96d0 msgr2=0x55e58c2d7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 -- 172.21.15.102:0/6554916 shutdown_connections 2024-04-24T23:22:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 -- 172.21.15.102:0/6554916 wait complete. 2024-04-24T23:22:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 -- start start 2024-04-24T23:22:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.207+0000 7f8b0f67d740 1 Processor -- start 2024-04-24T23:22:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0f67d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 0x55e58c2f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 0x55e58c2f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 0x55e58c2f1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36348/0 (socket says 172.21.15.102:36348) 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0659c640 1 -- 172.21.15.102:0/3517824691 learned_addr learned my addr 172.21.15.102:0/3517824691 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d5f20 0x55e58c2d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae800ceb0 con 0x55e58c2d5f20 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae800e320 con 0x55e58c2cdcd0 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae800d580 con 0x55e58c2cd3b0 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.208+0000 7f8b0f67d740 10 client.0 _ll_register_callbacks cb 0x7f8af8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.209+0000 7f8afffff640 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.209+0000 7f8b06d9d640 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d5f20 0x55e58c2d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.209+0000 7f8afffff640 1 -- 172.21.15.102:0/3517824691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 msgr2=0x55e58c2f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.209+0000 7f8afffff640 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 0x55e58c2f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.209+0000 7f8afffff640 1 -- 172.21.15.102:0/3517824691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d5f20 msgr2=0x55e58c2d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.209+0000 7f8afffff640 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d5f20 0x55e58c2d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.209+0000 7f8afffff640 1 -- 172.21.15.102:0/3517824691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e58c2d2c60 con 0x55e58c2cdcd0 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.210+0000 7f8afffff640 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d1eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8af00048b0 tx=0x7f8af0009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b588536b3ea8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.210+0000 7f8aedffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.210+0000 7f8aedffb640 1 -- 172.21.15.102:0/3517824691 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af009d050 con 0x55e58c2cdcd0 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.211+0000 7f8aedffb640 1 -- 172.21.15.102:0/3517824691 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7f8af0005510 con 0x55e58c2cdcd0 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.211+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af00d6070 con 0x55e58c2cdcd0 2024-04-24T23:22:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.211+0000 7f8aedffb640 1 -- 172.21.15.102:0/3517824691 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8af00d9070 con 0x55e58c2cdcd0 2024-04-24T23:22:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.211+0000 7f8b0f67d740 10 client.25577 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.211+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e58c360520 con 0x55e58c2cdcd0 2024-04-24T23:22:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.211+0000 7f8aedffb640 1 -- 172.21.15.102:0/3517824691 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8af009aef0 con 0x55e58c2cdcd0 2024-04-24T23:22:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.212+0000 7f8aedffb640 1 -- 172.21.15.102:0/3517824691 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8af0097560 con 0x55e58c2cdcd0 2024-04-24T23:22:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.212+0000 7f8b0f67d740 10 client.25577 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.212+0000 7f8b0f67d740 10 client.25577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.212+0000 7f8b0f67d740 20 client.25577 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.212+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55e58c3606f0 0x55e58c362ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.212+0000 7f8b0f67d740 4 client.25577 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:22:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.212+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8af00d6070 con 0x55e58c3606f0 2024-04-24T23:22:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.212+0000 7f8b0659c640 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55e58c3606f0 0x55e58c362ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.213+0000 7f8aedffb640 10 client.25577 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.213+0000 7f8b0659c640 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55e58c3606f0 0x55e58c362ae0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345e5b336bf43c0 server_cookie=227d57a69c16304e in_seq=0 out_seq=0 2024-04-24T23:22:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.227+0000 7f8aedffb640 1 -- 172.21.15.102:0/3517824691 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8af00d6070 con 0x55e58c3606f0 2024-04-24T23:22:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.227+0000 7f8aedffb640 10 client.25577 handle_command_reply: tid=0 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.227+0000 7f8b0f67d740 1 client.25577 shutdown 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.227+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 msgr2=0x55e58c2d1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.227+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d1eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8af00048b0 tx=0x7f8af0009740 comp rx=0 tx=0).stop 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 shutdown_connections 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55e58c3606f0 0x55e58c362ae0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e58c2d5f20 0x55e58c2d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e58c2cdcd0 0x55e58c2d1eb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8aedffb640 0 client.25577 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8b0f67d740 1 --2- 172.21.15.102:0/3517824691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e58c2cd3b0 0x55e58c2f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 >> 172.21.15.102:0/3517824691 conn(0x55e58c2c96d0 msgr2=0x55e58c2c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 shutdown_connections 2024-04-24T23:22:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8b0f67d740 1 -- 172.21.15.102:0/3517824691 wait complete. 2024-04-24T23:22:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.228+0000 7f8b0f67d740 20 client.25577 trim_cache size 0 max 16384 2024-04-24T23:22:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:22:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:22:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:22:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 -- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8af40a2fa0 msgr2=0x7f8af40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8af40a2fa0 0x7f8af40a5460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000a710 tx=0x7f8b0000d040 comp rx=0 tx=0).stop 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 -- 172.21.15.102:0/2929999956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b08100e20 msgr2=0x7f8b08109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b08100e20 0x7f8b08109ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8af000d6d0 tx=0x7f8af000db90 comp rx=0 tx=0).stop 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 -- 172.21.15.102:0/2929999956 shutdown_connections 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8af40a2fa0 0x7f8af40a5460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08106500 0x7f8b0810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b08100e20 0x7f8b08109ae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 --2- 172.21.15.102:0/2929999956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b08100460 0x7f8b081095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.229+0000 7f8b0c970640 1 -- 172.21.15.102:0/2929999956 >> 172.21.15.102:0/2929999956 conn(0x7f8b080fbf90 msgr2=0x7f8b080fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.230+0000 7f8b0c970640 1 -- 172.21.15.102:0/2929999956 shutdown_connections 2024-04-24T23:22:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:44.230+0000 7f8b0c970640 1 -- 172.21.15.102:0/2929999956 wait complete. 2024-04-24T23:22:45.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:44 smithi183 ceph-mon[33137]: pgmap v300: 97 pgs: 97 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 36 MiB/s wr, 9.20k op/s 2024-04-24T23:22:45.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:44 smithi102 ceph-mon[29216]: pgmap v300: 97 pgs: 97 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 36 MiB/s wr, 9.20k op/s 2024-04-24T23:22:45.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:44 smithi161 ceph-mon[32709]: pgmap v300: 97 pgs: 97 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 36 MiB/s wr, 9.20k op/s 2024-04-24T23:22:47.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:46 smithi183 ceph-mon[33137]: pgmap v301: 97 pgs: 97 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 24 MiB/s wr, 6.17k op/s 2024-04-24T23:22:47.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:46 smithi102 ceph-mon[29216]: pgmap v301: 97 pgs: 97 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 24 MiB/s wr, 6.17k op/s 2024-04-24T23:22:47.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:46 smithi161 ceph-mon[32709]: pgmap v301: 97 pgs: 97 active+clean; 14 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 24 MiB/s wr, 6.17k op/s 2024-04-24T23:22:49.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:48 smithi183 ceph-mon[33137]: pgmap v302: 97 pgs: 97 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 35 MiB/s wr, 8.80k op/s 2024-04-24T23:22:49.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:48 smithi102 ceph-mon[29216]: pgmap v302: 97 pgs: 97 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 35 MiB/s wr, 8.80k op/s 2024-04-24T23:22:49.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:48 smithi161 ceph-mon[32709]: pgmap v302: 97 pgs: 97 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 35 MiB/s wr, 8.80k op/s 2024-04-24T23:22:51.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:50 smithi183 ceph-mon[33137]: pgmap v303: 97 pgs: 97 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 25 MiB/s wr, 6.38k op/s 2024-04-24T23:22:51.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:50 smithi102 ceph-mon[29216]: pgmap v303: 97 pgs: 97 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 25 MiB/s wr, 6.38k op/s 2024-04-24T23:22:51.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:50 smithi161 ceph-mon[32709]: pgmap v303: 97 pgs: 97 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 25 MiB/s wr, 6.38k op/s 2024-04-24T23:22:53.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:52 smithi183 ceph-mon[33137]: pgmap v304: 97 pgs: 97 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 24 MiB/s wr, 6.03k op/s 2024-04-24T23:22:53.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:52 smithi102 ceph-mon[29216]: pgmap v304: 97 pgs: 97 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 24 MiB/s wr, 6.03k op/s 2024-04-24T23:22:53.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:52 smithi161 ceph-mon[32709]: pgmap v304: 97 pgs: 97 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 24 MiB/s wr, 6.03k op/s 2024-04-24T23:22:55.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:54 smithi183 ceph-mon[33137]: pgmap v305: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 35 MiB/s wr, 8.90k op/s 2024-04-24T23:22:55.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:54 smithi102 ceph-mon[29216]: pgmap v305: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 35 MiB/s wr, 8.90k op/s 2024-04-24T23:22:55.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:54 smithi161 ceph-mon[32709]: pgmap v305: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 35 MiB/s wr, 8.90k op/s 2024-04-24T23:22:57.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:56 smithi183 ceph-mon[33137]: pgmap v306: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 5.83k op/s 2024-04-24T23:22:57.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:56 smithi102 ceph-mon[29216]: pgmap v306: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 5.83k op/s 2024-04-24T23:22:57.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:56 smithi161 ceph-mon[32709]: pgmap v306: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 23 MiB/s wr, 5.83k op/s 2024-04-24T23:22:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7efffbfff640 1 -- 172.21.15.102:0/2932511657 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000033600 con 0x7f00040ffe20 2024-04-24T23:22:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7f000a3cf640 1 -- 172.21.15.102:0/2932511657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00040ffe20 msgr2=0x7f0004100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2932511657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00040ffe20 0x7f0004100200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f000000c8c0 tx=0x7f000002c670 comp rx=0 tx=0).stop 2024-04-24T23:22:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7f000a3cf640 1 -- 172.21.15.102:0/2932511657 shutdown_connections 2024-04-24T23:22:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2932511657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0004104ef0 0x7f000410cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2932511657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00041007d0 0x7f0004104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2932511657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00040ffe20 0x7f0004100200 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7f000a3cf640 1 -- 172.21.15.102:0/2932511657 >> 172.21.15.102:0/2932511657 conn(0x7f00040769c0 msgr2=0x7f00040fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7f000a3cf640 1 -- 172.21.15.102:0/2932511657 shutdown_connections 2024-04-24T23:22:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.377+0000 7f000a3cf640 1 -- 172.21.15.102:0/2932511657 wait complete. 2024-04-24T23:22:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.378+0000 7f000a3cf640 1 Processor -- start 2024-04-24T23:22:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.378+0000 7f000a3cf640 1 -- start start 2024-04-24T23:22:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.378+0000 7f000a3cf640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00040ffe20 0x7f000410b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.379+0000 7f000a3cf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004104ef0 0x7f000410baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.379+0000 7f000a3cf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00041a7f60 0x7f00041aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.379+0000 7f000a3cf640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f000410ec20 con 0x7f00040ffe20 2024-04-24T23:22:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.379+0000 7f000a3cf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f000410eaa0 con 0x7f00041a7f60 2024-04-24T23:22:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.379+0000 7f000a3cf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f000410eda0 con 0x7f0004104ef0 2024-04-24T23:22:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.379+0000 7f0008bcc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004104ef0 0x7f000410baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.379+0000 7f0008bcc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004104ef0 0x7f000410baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:45992/0 (socket says 172.21.15.102:45992) 2024-04-24T23:22:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.379+0000 7f0008bcc640 1 -- 172.21.15.102:0/2503545210 learned_addr learned my addr 172.21.15.102:0/2503545210 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.379+0000 7f00093cd640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00040ffe20 0x7f000410b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.380+0000 7f0009bce640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00041a7f60 0x7f00041aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.380+0000 7f0008bcc640 1 -- 172.21.15.102:0/2503545210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00041a7f60 msgr2=0x7f00041aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.380+0000 7f0008bcc640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00041a7f60 0x7f00041aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.380+0000 7f0008bcc640 1 -- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00040ffe20 msgr2=0x7f000410b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.380+0000 7f0008bcc640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00040ffe20 0x7f000410b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.380+0000 7f0008bcc640 1 -- 172.21.15.102:0/2503545210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00041aaad0 con 0x7f0004104ef0 2024-04-24T23:22:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.380+0000 7f0009bce640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00041a7f60 0x7f00041aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.380+0000 7f0008bcc640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004104ef0 0x7f000410baa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efff400e9f0 tx=0x7efff400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.381+0000 7efffa7fc640 1 -- 172.21.15.102:0/2503545210 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff400ccc0 con 0x7f0004104ef0 2024-04-24T23:22:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.381+0000 7efffa7fc640 1 -- 172.21.15.102:0/2503545210 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efff40108e0 con 0x7f0004104ef0 2024-04-24T23:22:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.381+0000 7f000a3cf640 1 -- 172.21.15.102:0/2503545210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00041aadc0 con 0x7f0004104ef0 2024-04-24T23:22:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.381+0000 7f000a3cf640 1 -- 172.21.15.102:0/2503545210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00041ab280 con 0x7f0004104ef0 2024-04-24T23:22:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.382+0000 7efffa7fc640 1 -- 172.21.15.102:0/2503545210 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4018880 con 0x7f0004104ef0 2024-04-24T23:22:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.385+0000 7f000e418740 1 Processor -- start 2024-04-24T23:22:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.385+0000 7f000e418740 1 -- start start 2024-04-24T23:22:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.385+0000 7f000e418740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502ce7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.385+0000 7f000e418740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce7cd0 0x558502cf1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f000e418740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502cf23a0 0x558502cf4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f000e418740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558502c4bcb0 con 0x558502cf23a0 2024-04-24T23:22:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f000e418740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558502bf45b0 con 0x558502ce73b0 2024-04-24T23:22:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f000e418740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558502c47c70 con 0x558502ce7cd0 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f0009bce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502ce7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f00093cd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce7cd0 0x558502cf1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f0009bce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502ce7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:39150/0 (socket says 172.21.15.102:39150) 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f0009bce640 1 -- 172.21.15.102:0/2584801051 learned_addr learned my addr 172.21.15.102:0/2584801051 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7efffa7fc640 1 -- 172.21.15.102:0/2503545210 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7efff4018a20 con 0x7f0004104ef0 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f0009bce640 1 -- 172.21.15.102:0/2584801051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce7cd0 msgr2=0x558502cf1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f0009bce640 1 --2- 172.21.15.102:0/2584801051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce7cd0 0x558502cf1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f0009bce640 1 -- 172.21.15.102:0/2584801051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502cf23a0 msgr2=0x558502cf4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f0009bce640 1 --2- 172.21.15.102:0/2584801051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502cf23a0 0x558502cf4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f0009bce640 1 -- 172.21.15.102:0/2584801051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558502960210 con 0x558502ce73b0 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7f0009bce640 1 --2- 172.21.15.102:0/2584801051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502ce7790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efffc010710 tx=0x7efffc010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c5b2ce1da7c478 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.386+0000 7effef7fe640 1 -- 172.21.15.102:0/2584801051 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc00e8d0 con 0x558502ce73b0 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.387+0000 7effef7fe640 1 -- 172.21.15.102:0/2584801051 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efffc012450 con 0x558502ce73b0 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.387+0000 7efffa7fc640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efff00a2e40 0x7efff00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.387+0000 7effef7fe640 1 -- 172.21.15.102:0/2584801051 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc01c670 con 0x558502ce73b0 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.387+0000 7f00093cd640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efff00a2e40 0x7efff00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.387+0000 7efffa7fc640 1 -- 172.21.15.102:0/2503545210 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7efff40cf550 con 0x7f0004104ef0 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 -- 172.21.15.102:0/2584801051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 msgr2=0x558502ce7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 --2- 172.21.15.102:0/2584801051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502ce7790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efffc010710 tx=0x7efffc010bd0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 -- 172.21.15.102:0/2584801051 shutdown_connections 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 --2- 172.21.15.102:0/2584801051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502cf23a0 0x558502cf4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 --2- 172.21.15.102:0/2584801051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce7cd0 0x558502cf1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 --2- 172.21.15.102:0/2584801051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502ce7790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 -- 172.21.15.102:0/2584801051 >> 172.21.15.102:0/2584801051 conn(0x558502ce36d0 msgr2=0x558502ce3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f00093cd640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efff00a2e40 0x7efff00a5300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0000005b60 tx=0x7f0000005a90 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 -- 172.21.15.102:0/2584801051 shutdown_connections 2024-04-24T23:22:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 -- 172.21.15.102:0/2584801051 wait complete. 2024-04-24T23:22:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 -- start start 2024-04-24T23:22:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.388+0000 7f000e418740 1 Processor -- start 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f000e418740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce73b0 0x558502cef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f000e418740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 0x558502cef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f000e418740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502cf23a0 0x558502cefdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f000e418740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558502cf5190 con 0x558502ce7cd0 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f000e418740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558502c07b50 con 0x558502cf23a0 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f000e418740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558502cf5310 con 0x558502ce73b0 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f000e418740 10 client.0 _ll_register_callbacks cb 0x558502c42ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f00093cd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce73b0 0x558502cef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f00093cd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce73b0 0x558502cef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:46008/0 (socket says 172.21.15.102:46008) 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f00093cd640 1 -- 172.21.15.102:0/1372793889 learned_addr learned my addr 172.21.15.102:0/1372793889 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.389+0000 7f0009bce640 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502cf23a0 0x558502cefdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7f0008bcc640 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 0x558502cef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7f0009bce640 1 -- 172.21.15.102:0/1372793889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce73b0 msgr2=0x558502cef370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7f0009bce640 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce73b0 0x558502cef370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7f0009bce640 1 -- 172.21.15.102:0/1372793889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 msgr2=0x558502cef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7f0009bce640 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 0x558502cef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7f0009bce640 1 -- 172.21.15.102:0/1372793889 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558502a2c620 con 0x558502cf23a0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7f0009bce640 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502cf23a0 0x558502cefdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efffc000f80 tx=0x7efffc012a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6faeca31b9ff819c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7effecff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7effecff9640 1 -- 172.21.15.102:0/1372793889 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc00ece0 con 0x558502cf23a0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7effecff9640 1 -- 172.21.15.102:0/1372793889 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7efffc01c410 con 0x558502cf23a0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.390+0000 7f000e418740 1 -- 172.21.15.102:0/1372793889 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efffc012450 con 0x558502cf23a0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7effecff9640 1 -- 172.21.15.102:0/1372793889 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efffc033050 con 0x558502cf23a0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7f000e418740 10 client.25589 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7f000e418740 1 -- 172.21.15.102:0/1372793889 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558502d01460 con 0x558502cf23a0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7effecff9640 1 -- 172.21.15.102:0/1372793889 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efffc012450 con 0x558502cf23a0 2024-04-24T23:22:57.395 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7effecff9640 1 -- 172.21.15.102:0/1372793889 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efffc02e030 con 0x558502cf23a0 2024-04-24T23:22:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7f000e418740 10 client.25589 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7f000e418740 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7f000e418740 20 client.25589 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7f000e418740 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558502d028c0 0x558502d04cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7f000e418740 4 client.25589 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:22:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7f000e418740 1 -- 172.21.15.102:0/1372793889 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558502c591f0 con 0x558502d028c0 2024-04-24T23:22:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.391+0000 7f00093cd640 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558502d028c0 0x558502d04cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.392+0000 7effecff9640 10 client.25589 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.392+0000 7f00093cd640 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558502d028c0 0x558502d04cb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4915f2875cc67f server_cookie=85efd67af217457c in_seq=0 out_seq=0 2024-04-24T23:22:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.393+0000 7effecff9640 1 -- 172.21.15.102:0/1372793889 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558502c591f0 con 0x558502d028c0 2024-04-24T23:22:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.393+0000 7effecff9640 10 client.25589 handle_command_reply: tid=0 2024-04-24T23:22:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.393+0000 7f000e418740 1 client.25589 shutdown 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.393+0000 7f000e418740 1 -- 172.21.15.102:0/1372793889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502cf23a0 msgr2=0x558502cefdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.393+0000 7f000e418740 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502cf23a0 0x558502cefdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efffc000f80 tx=0x7efffc012a30 comp rx=0 tx=0).stop 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.394+0000 7f000e418740 1 -- 172.21.15.102:0/1372793889 shutdown_connections 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.394+0000 7f000e418740 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502cf23a0 0x558502cefdf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.394+0000 7f000e418740 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558502d028c0 0x558502d04cb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.394+0000 7f000e418740 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 0x558502cef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.394+0000 7f000e418740 1 --2- 172.21.15.102:0/1372793889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502ce73b0 0x558502cef370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.394+0000 7f000e418740 1 -- 172.21.15.102:0/1372793889 >> 172.21.15.102:0/1372793889 conn(0x558502ce36d0 msgr2=0x558502ce93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.394+0000 7effecff9640 0 client.25589 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.394+0000 7f000e418740 1 -- 172.21.15.102:0/1372793889 shutdown_connections 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.394+0000 7f000e418740 1 -- 172.21.15.102:0/1372793889 wait complete. 2024-04-24T23:22:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.395+0000 7f000e418740 20 client.25589 trim_cache size 0 max 16384 2024-04-24T23:22:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.426+0000 7f000e418740 1 Processor -- start 2024-04-24T23:22:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.426+0000 7f000e418740 1 -- start start 2024-04-24T23:22:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.426+0000 7f000e418740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce73b0 0x558502cf1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.426+0000 7f000e418740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce7cd0 0x558502cfe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.426+0000 7f000e418740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 0x558502d0a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.426+0000 7f000e418740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7effe8011120 con 0x558502ce73b0 2024-04-24T23:22:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.426+0000 7f000e418740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7effe800e9f0 con 0x558502ce7cd0 2024-04-24T23:22:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.426+0000 7f000e418740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7effe800c860 con 0x558502cf23a0 2024-04-24T23:22:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f00093cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce73b0 0x558502cf1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f00093cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce73b0 0x558502cf1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40064/0 (socket says 172.21.15.102:40064) 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f00093cd640 1 -- 172.21.15.102:0/1156208886 learned_addr learned my addr 172.21.15.102:0/1156208886 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f0008bcc640 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce7cd0 0x558502cfe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f0009bce640 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 0x558502d0a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f00093cd640 1 -- 172.21.15.102:0/1156208886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 msgr2=0x558502d0a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f00093cd640 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 0x558502d0a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f00093cd640 1 -- 172.21.15.102:0/1156208886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce7cd0 msgr2=0x558502cfe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f00093cd640 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce7cd0 0x558502cfe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f00093cd640 1 -- 172.21.15.102:0/1156208886 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558502cf02d0 con 0x558502ce73b0 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.427+0000 7f0008bcc640 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce7cd0 0x558502cfe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.428+0000 7f00093cd640 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce73b0 0x558502cf1aa0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c00 tx=0x7f0000040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78aa8a09a5f0c8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.428+0000 7effeeffd640 1 -- 172.21.15.102:0/1156208886 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000057070 con 0x558502ce73b0 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.428+0000 7effeeffd640 1 -- 172.21.15.102:0/1156208886 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0000042070 con 0x558502ce73b0 2024-04-24T23:22:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.428+0000 7effeeffd640 1 -- 172.21.15.102:0/1156208886 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000004dd0 con 0x558502ce73b0 2024-04-24T23:22:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.429+0000 7f000e418740 1 -- 172.21.15.102:0/1156208886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce73b0 msgr2=0x558502cf1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.429+0000 7f000e418740 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce73b0 0x558502cf1aa0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c00 tx=0x7f0000040000 comp rx=0 tx=0).stop 2024-04-24T23:22:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.429+0000 7f000e418740 1 -- 172.21.15.102:0/1156208886 shutdown_connections 2024-04-24T23:22:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.429+0000 7f000e418740 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 0x558502d0a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.429+0000 7f000e418740 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce7cd0 0x558502cfe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.429+0000 7f000e418740 1 --2- 172.21.15.102:0/1156208886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce73b0 0x558502cf1aa0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.429+0000 7f000e418740 1 -- 172.21.15.102:0/1156208886 >> 172.21.15.102:0/1156208886 conn(0x558502ce36d0 msgr2=0x558502cefbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.429+0000 7f000e418740 1 -- 172.21.15.102:0/1156208886 shutdown_connections 2024-04-24T23:22:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.429+0000 7f000e418740 1 -- 172.21.15.102:0/1156208886 wait complete. 2024-04-24T23:22:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.430+0000 7f000e418740 1 -- start start 2024-04-24T23:22:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.430+0000 7f000e418740 1 Processor -- start 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f000e418740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502d61390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f000e418740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 0x558502d618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f000e418740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 0x558502cf95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f000e418740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7effe800ceb0 con 0x558502ce7cd0 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f000e418740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7effe800e320 con 0x558502ce73b0 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f000e418740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7effe800d580 con 0x558502cf23a0 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f000e418740 10 client.0 _ll_register_callbacks cb 0x7efffc016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f00093cd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502d61390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f00093cd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502d61390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:39170/0 (socket says 172.21.15.102:39170) 2024-04-24T23:22:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f00093cd640 1 -- 172.21.15.102:0/589993690 learned_addr learned my addr 172.21.15.102:0/589993690 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:22:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f0009bce640 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 0x558502cf95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f0008bcc640 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 0x558502d618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f0009bce640 1 -- 172.21.15.102:0/589993690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 msgr2=0x558502d61390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f0009bce640 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502d61390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f0009bce640 1 -- 172.21.15.102:0/589993690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 msgr2=0x558502d618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:22:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f0009bce640 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 0x558502d618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.431+0000 7f0009bce640 1 -- 172.21.15.102:0/589993690 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558502cf9b20 con 0x558502cf23a0 2024-04-24T23:22:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.432+0000 7f0009bce640 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 0x558502cf95e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efffc011010 tx=0x7efffc0268a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45181685cc7e208f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.432+0000 7effecff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:22:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.432+0000 7effecff9640 1 -- 172.21.15.102:0/589993690 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc03b020 con 0x558502cf23a0 2024-04-24T23:22:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.432+0000 7effecff9640 1 -- 172.21.15.102:0/589993690 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8579+0+0 (secure 0 0 0) 0x7efffc00ece0 con 0x558502cf23a0 2024-04-24T23:22:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.432+0000 7f000e418740 1 -- 172.21.15.102:0/589993690 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0000042070 con 0x558502cf23a0 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.432+0000 7effecff9640 1 -- 172.21.15.102:0/589993690 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efffc046050 con 0x558502cf23a0 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.432+0000 7f000e418740 10 client.25525 fetch_fsmap learned FSMap version 26 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.432+0000 7f000e418740 1 -- 172.21.15.102:0/589993690 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558502cfada0 con 0x558502cf23a0 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.433+0000 7effecff9640 1 -- 172.21.15.102:0/589993690 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0000042070 con 0x558502cf23a0 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.433+0000 7effecff9640 1 -- 172.21.15.102:0/589993690 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efffc02e040 con 0x558502cf23a0 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.433+0000 7f000e418740 10 client.25525 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.433+0000 7f000e418740 10 client.25525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.433+0000 7f000e418740 20 client.25525 populate_metadata read hostname 'smithi102' 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.433+0000 7f000e418740 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558502d8af70 0x558502d8d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.433+0000 7f000e418740 4 client.25525 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.433+0000 7f000e418740 1 -- 172.21.15.102:0/589993690 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558502c53540 con 0x558502d8af70 2024-04-24T23:22:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.434+0000 7f00093cd640 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558502d8af70 0x558502d8d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.434+0000 7f00093cd640 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558502d8af70 0x558502d8d390 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa286826fcfde302 server_cookie=25b149849ddeb7c3 in_seq=0 out_seq=0 2024-04-24T23:22:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.434+0000 7effecff9640 10 client.25525 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7effecff9640 1 -- 172.21.15.102:0/589993690 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558502c53540 con 0x558502d8af70 2024-04-24T23:22:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7effecff9640 10 client.25525 handle_command_reply: tid=0 2024-04-24T23:22:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7f000e418740 1 client.25525 shutdown 2024-04-24T23:22:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7f000e418740 1 -- 172.21.15.102:0/589993690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 msgr2=0x558502cf95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7f000e418740 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 0x558502cf95e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efffc011010 tx=0x7efffc0268a0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7f000e418740 1 -- 172.21.15.102:0/589993690 shutdown_connections 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7f000e418740 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558502cf23a0 0x558502cf95e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7f000e418740 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558502d8af70 0x558502d8d390 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7f000e418740 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558502ce7cd0 0x558502d618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7f000e418740 1 --2- 172.21.15.102:0/589993690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558502ce73b0 0x558502d61390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7f000e418740 1 -- 172.21.15.102:0/589993690 >> 172.21.15.102:0/589993690 conn(0x558502ce36d0 msgr2=0x558502ce8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.445+0000 7effecff9640 0 client.25525 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.446+0000 7f000e418740 1 -- 172.21.15.102:0/589993690 shutdown_connections 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.446+0000 7f000e418740 1 -- 172.21.15.102:0/589993690 wait complete. 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.446+0000 7f000e418740 20 client.25525 trim_cache size 0 max 16384 2024-04-24T23:22:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:22:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:22:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:22:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:22:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.446+0000 7f000a3cf640 1 -- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efff00a2e40 msgr2=0x7efff00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.446+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efff00a2e40 0x7efff00a5300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0000005b60 tx=0x7f0000005a90 comp rx=0 tx=0).stop 2024-04-24T23:22:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.446+0000 7f000a3cf640 1 -- 172.21.15.102:0/2503545210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004104ef0 msgr2=0x7f000410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.446+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004104ef0 0x7f000410baa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efff400e9f0 tx=0x7efff400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.447+0000 7f000a3cf640 1 -- 172.21.15.102:0/2503545210 shutdown_connections 2024-04-24T23:22:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.447+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00041a7f60 0x7f00041aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.447+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004104ef0 0x7f000410baa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.447+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efff00a2e40 0x7efff00a5300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.447+0000 7f000a3cf640 1 --2- 172.21.15.102:0/2503545210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00040ffe20 0x7f000410b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.447+0000 7f000a3cf640 1 -- 172.21.15.102:0/2503545210 >> 172.21.15.102:0/2503545210 conn(0x7f00040769c0 msgr2=0x7f0004111410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.447+0000 7f000a3cf640 1 -- 172.21.15.102:0/2503545210 shutdown_connections 2024-04-24T23:22:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:22:57.447+0000 7f000a3cf640 1 -- 172.21.15.102:0/2503545210 wait complete. 2024-04-24T23:22:59.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:22:58 smithi183 ceph-mon[33137]: pgmap v307: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 33 MiB/s wr, 8.46k op/s 2024-04-24T23:22:59.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:22:58 smithi102 ceph-mon[29216]: pgmap v307: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 33 MiB/s wr, 8.46k op/s 2024-04-24T23:22:59.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:22:58 smithi161 ceph-mon[32709]: pgmap v307: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 33 MiB/s wr, 8.46k op/s 2024-04-24T23:23:01.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:01 smithi183 ceph-mon[33137]: pgmap v308: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 24 MiB/s wr, 6.21k op/s 2024-04-24T23:23:01.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:01 smithi102 ceph-mon[29216]: pgmap v308: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 24 MiB/s wr, 6.21k op/s 2024-04-24T23:23:01.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:01 smithi161 ceph-mon[32709]: pgmap v308: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 24 MiB/s wr, 6.21k op/s 2024-04-24T23:23:03.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:03 smithi183 ceph-mon[33137]: pgmap v309: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 23 MiB/s wr, 5.88k op/s 2024-04-24T23:23:03.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:03 smithi102 ceph-mon[29216]: pgmap v309: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 23 MiB/s wr, 5.88k op/s 2024-04-24T23:23:03.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:03 smithi161 ceph-mon[32709]: pgmap v309: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 23 MiB/s wr, 5.88k op/s 2024-04-24T23:23:05.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:05 smithi102 ceph-mon[29216]: pgmap v310: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 33 MiB/s wr, 8.43k op/s 2024-04-24T23:23:05.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:05 smithi161 ceph-mon[32709]: pgmap v310: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 33 MiB/s wr, 8.43k op/s 2024-04-24T23:23:05.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:05 smithi183 ceph-mon[33137]: pgmap v310: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 33 MiB/s wr, 8.43k op/s 2024-04-24T23:23:07.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:07 smithi102 ceph-mon[29216]: pgmap v311: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 22 MiB/s wr, 5.55k op/s 2024-04-24T23:23:07.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:07 smithi102 ceph-mon[29216]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T23:23:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:07 smithi161 ceph-mon[32709]: pgmap v311: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 22 MiB/s wr, 5.55k op/s 2024-04-24T23:23:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:07 smithi161 ceph-mon[32709]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T23:23:07.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:07 smithi183 ceph-mon[33137]: pgmap v311: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 22 MiB/s wr, 5.55k op/s 2024-04-24T23:23:07.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:07 smithi183 ceph-mon[33137]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T23:23:09.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:09 smithi102 ceph-mon[29216]: pgmap v313: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 22 MiB/s wr, 5.54k op/s 2024-04-24T23:23:09.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:09 smithi161 ceph-mon[32709]: pgmap v313: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 22 MiB/s wr, 5.54k op/s 2024-04-24T23:23:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:09 smithi183 ceph-mon[33137]: pgmap v313: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 22 MiB/s wr, 5.54k op/s 2024-04-24T23:23:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.552+0000 7f3496c09640 1 -- 172.21.15.102:0/2463155729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3490101730 msgr2=0x7f3490101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.552+0000 7f3496c09640 1 --2- 172.21.15.102:0/2463155729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3490101730 0x7f3490101bb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3480009cd0 tx=0x7f348002a8e0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.552+0000 7f3496c09640 1 -- 172.21.15.102:0/2463155729 shutdown_connections 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.552+0000 7f3496c09640 1 --2- 172.21.15.102:0/2463155729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34901020f0 0x7f349010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.552+0000 7f3496c09640 1 --2- 172.21.15.102:0/2463155729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3490101730 0x7f3490101bb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.552+0000 7f3496c09640 1 --2- 172.21.15.102:0/2463155729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3490107740 0x7f3490107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.552+0000 7f3496c09640 1 -- 172.21.15.102:0/2463155729 >> 172.21.15.102:0/2463155729 conn(0x7f34900fd200 msgr2=0x7f34900ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.552+0000 7f3496c09640 1 -- 172.21.15.102:0/2463155729 shutdown_connections 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.552+0000 7f3496c09640 1 -- 172.21.15.102:0/2463155729 wait complete. 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f3496c09640 1 Processor -- start 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f3496c09640 1 -- start start 2024-04-24T23:23:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f3496c09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3490101730 0x7f3490197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f3496c09640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34901020f0 0x7f3490198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f3496c09640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3490107740 0x7f349019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f3496c09640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3490111a00 con 0x7f34901020f0 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f3496c09640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3490111880 con 0x7f3490107740 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f3496c09640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3490111b80 con 0x7f3490101730 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f348ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34901020f0 0x7f3490198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f348ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34901020f0 0x7f3490198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48046/0 (socket says 172.21.15.102:48046) 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f348ffff640 1 -- 172.21.15.102:0/2539924211 learned_addr learned my addr 172.21.15.102:0/2539924211 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f34951a6640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3490107740 0x7f349019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f34949a5640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3490101730 0x7f3490197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f348ffff640 1 -- 172.21.15.102:0/2539924211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3490101730 msgr2=0x7f3490197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f348ffff640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3490101730 0x7f3490197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f348ffff640 1 -- 172.21.15.102:0/2539924211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3490107740 msgr2=0x7f349019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f348ffff640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3490107740 0x7f349019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.553+0000 7f348ffff640 1 -- 172.21.15.102:0/2539924211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349019cee0 con 0x7f34901020f0 2024-04-24T23:23:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.554+0000 7f34951a6640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3490107740 0x7f349019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.554+0000 7f348ffff640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34901020f0 0x7f3490198480 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3480006010 tx=0x7f3480003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.554+0000 7f348dffb640 1 -- 172.21.15.102:0/2539924211 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480004450 con 0x7f34901020f0 2024-04-24T23:23:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.554+0000 7f348dffb640 1 -- 172.21.15.102:0/2539924211 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34800045f0 con 0x7f34901020f0 2024-04-24T23:23:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.554+0000 7f3496c09640 1 -- 172.21.15.102:0/2539924211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349019d170 con 0x7f34901020f0 2024-04-24T23:23:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.554+0000 7f348dffb640 1 -- 172.21.15.102:0/2539924211 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800466e0 con 0x7f34901020f0 2024-04-24T23:23:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.554+0000 7f3496c09640 1 -- 172.21.15.102:0/2539924211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34901a4a60 con 0x7f34901020f0 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.556+0000 7f348dffb640 1 -- 172.21.15.102:0/2539924211 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f34800305a0 con 0x7f34901020f0 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34999f1740 1 Processor -- start 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34999f1740 1 -- start start 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34999f1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49240 0x55b0fef49620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f348dffb640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f347c0a2e20 0x7f347c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34999f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49b60 0x55b0fef53cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34999f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef56620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34999f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b0feeadcb0 con 0x55b0fef54230 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34999f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b0fee565b0 con 0x55b0fef49b60 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34999f1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0feea9c70 con 0x55b0fef49240 2024-04-24T23:23:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef56620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34949a5640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f347c0a2e20 0x7f347c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f348dffb640 1 -- 172.21.15.102:0/2539924211 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8623+0+0 (secure 0 0 0) 0x7f348002e030 con 0x7f34901020f0 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef56620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48060/0 (socket says 172.21.15.102:48060) 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 -- 172.21.15.102:0/2043763847 learned_addr learned my addr 172.21.15.102:0/2043763847 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f348ffff640 1 --2- 172.21.15.102:0/2043763847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49b60 0x55b0fef53cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 --2- 172.21.15.102:0/2043763847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49240 0x55b0fef49620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 -- 172.21.15.102:0/2043763847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49240 msgr2=0x55b0fef49620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 --2- 172.21.15.102:0/2043763847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49240 0x55b0fef49620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 -- 172.21.15.102:0/2043763847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49b60 msgr2=0x55b0fef53cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 --2- 172.21.15.102:0/2043763847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49b60 0x55b0fef53cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 -- 172.21.15.102:0/2043763847 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0febc2210 con 0x55b0fef54230 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34949a5640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f347c0a2e20 0x7f347c0a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3488009990 tx=0x7f3488009110 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.557+0000 7f34951a6640 1 --2- 172.21.15.102:0/2043763847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef56620 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3478005fd0 tx=0x7f347800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3822fd9a21e41b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f3486ffd640 1 -- 172.21.15.102:0/2043763847 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478004030 con 0x55b0fef54230 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f3486ffd640 1 -- 172.21.15.102:0/2043763847 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f347800e960 con 0x55b0fef54230 2024-04-24T23:23:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f3486ffd640 1 -- 172.21.15.102:0/2043763847 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780038c0 con 0x55b0fef54230 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f34999f1740 1 -- 172.21.15.102:0/2043763847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 msgr2=0x55b0fef56620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f34999f1740 1 --2- 172.21.15.102:0/2043763847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef56620 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3478005fd0 tx=0x7f347800b4c0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f34999f1740 1 -- 172.21.15.102:0/2043763847 shutdown_connections 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f34999f1740 1 --2- 172.21.15.102:0/2043763847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef56620 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f34999f1740 1 --2- 172.21.15.102:0/2043763847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49b60 0x55b0fef53cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f34999f1740 1 --2- 172.21.15.102:0/2043763847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49240 0x55b0fef49620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f34999f1740 1 -- 172.21.15.102:0/2043763847 >> 172.21.15.102:0/2043763847 conn(0x55b0fef45560 msgr2=0x55b0fef45940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f34999f1740 1 -- 172.21.15.102:0/2043763847 shutdown_connections 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.558+0000 7f34999f1740 1 -- 172.21.15.102:0/2043763847 wait complete. 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34999f1740 1 -- start start 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34999f1740 1 Processor -- start 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34999f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fef511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34999f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 0x55b0fef516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34999f1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 0x55b0fef51c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34999f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b0fef57020 con 0x55b0fef49b60 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34999f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b0fee69b50 con 0x55b0fef49240 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34999f1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0fef571a0 con 0x55b0fef54230 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34999f1740 10 client.0 _ll_register_callbacks cb 0x55b0feea4ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f348ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 0x55b0fef516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f348ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 0x55b0fef516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48076/0 (socket says 172.21.15.102:48076) 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f348ffff640 1 -- 172.21.15.102:0/1126400460 learned_addr learned my addr 172.21.15.102:0/1126400460 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34949a5640 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fef511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f34951a6640 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 0x55b0fef51c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f348ffff640 1 -- 172.21.15.102:0/1126400460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 msgr2=0x55b0fef51c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f348ffff640 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 0x55b0fef51c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f348ffff640 1 -- 172.21.15.102:0/1126400460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 msgr2=0x55b0fef511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f348ffff640 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fef511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.559+0000 7f348ffff640 1 -- 172.21.15.102:0/1126400460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fec8e620 con 0x55b0fef49b60 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f348ffff640 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 0x55b0fef516f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f34800ee950 tx=0x7f34800bcbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af3607854b9f6d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f346ffff640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f346ffff640 1 -- 172.21.15.102:0/1126400460 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800098d0 con 0x55b0fef49b60 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f346ffff640 1 -- 172.21.15.102:0/1126400460 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8623+0+0 (secure 0 0 0) 0x7f34800bb530 con 0x55b0fef49b60 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34999f1740 1 -- 172.21.15.102:0/1126400460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f347800e960 con 0x55b0fef49b60 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f346ffff640 1 -- 172.21.15.102:0/1126400460 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34800bd930 con 0x55b0fef49b60 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34999f1740 10 client.16206 fetch_fsmap learned FSMap version 26 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34999f1740 1 -- 172.21.15.102:0/1126400460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0fef63200 con 0x55b0fef49b60 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f346ffff640 1 -- 172.21.15.102:0/1126400460 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f347800e960 con 0x55b0fef49b60 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f346ffff640 1 -- 172.21.15.102:0/1126400460 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f348004e070 con 0x55b0fef49b60 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34999f1740 10 client.16206 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34999f1740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34999f1740 20 client.16206 populate_metadata read hostname 'smithi102' 2024-04-24T23:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34999f1740 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55b0fef64660 0x55b0fef66ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34999f1740 4 client.16206 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34999f1740 1 -- 172.21.15.102:0/1126400460 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0feebb1f0 con 0x55b0fef64660 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.560+0000 7f34949a5640 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55b0fef64660 0x55b0fef66ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.561+0000 7f346ffff640 10 client.16206 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.561+0000 7f34949a5640 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55b0fef64660 0x55b0fef66ad0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecdff86094daf281 server_cookie=3c3817401c1d7b37 in_seq=0 out_seq=0 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.561+0000 7f346ffff640 1 -- 172.21.15.102:0/1126400460 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0feebb1f0 con 0x55b0fef64660 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.561+0000 7f346ffff640 10 client.16206 handle_command_reply: tid=0 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.561+0000 7f34999f1740 1 client.16206 shutdown 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 -- 172.21.15.102:0/1126400460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 msgr2=0x55b0fef516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 0x55b0fef516f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f34800ee950 tx=0x7f34800bcbb0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 -- 172.21.15.102:0/1126400460 shutdown_connections 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 0x55b0fef51c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55b0fef64660 0x55b0fef66ad0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 0x55b0fef516f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f346ffff640 0 client.16206 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 --2- 172.21.15.102:0/1126400460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fef511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 -- 172.21.15.102:0/1126400460 >> 172.21.15.102:0/1126400460 conn(0x55b0fef45560 msgr2=0x55b0fef4b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 -- 172.21.15.102:0/1126400460 shutdown_connections 2024-04-24T23:23:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 1 -- 172.21.15.102:0/1126400460 wait complete. 2024-04-24T23:23:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.562+0000 7f34999f1740 20 client.16206 trim_cache size 0 max 16384 2024-04-24T23:23:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34999f1740 1 Processor -- start 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34999f1740 1 -- start start 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34999f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fef53870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34999f1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49b60 0x55b0fef4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34999f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef44440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34999f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34901afc20 con 0x55b0fef54230 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34999f1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34901ad4f0 con 0x55b0fef49240 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34999f1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34901ab360 con 0x55b0fef49b60 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34951a6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef44440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34951a6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef44440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48092/0 (socket says 172.21.15.102:48092) 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34951a6640 1 -- 172.21.15.102:0/1976670931 learned_addr learned my addr 172.21.15.102:0/1976670931 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f34949a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fef53870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.580+0000 7f348ffff640 1 --2- 172.21.15.102:0/1976670931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49b60 0x55b0fef4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.581+0000 7f34951a6640 1 -- 172.21.15.102:0/1976670931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49b60 msgr2=0x55b0fef4b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.581+0000 7f34951a6640 1 --2- 172.21.15.102:0/1976670931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49b60 0x55b0fef4b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.581+0000 7f34951a6640 1 -- 172.21.15.102:0/1976670931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 msgr2=0x55b0fef53870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.581+0000 7f34951a6640 1 --2- 172.21.15.102:0/1976670931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fef53870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.581+0000 7f34951a6640 1 -- 172.21.15.102:0/1976670931 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fef49990 con 0x55b0fef54230 2024-04-24T23:23:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.581+0000 7f34951a6640 1 --2- 172.21.15.102:0/1976670931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef44440 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f347800e720 tx=0x7f3478005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbf3653c6158228 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.581+0000 7f34867fc640 1 -- 172.21.15.102:0/1976670931 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780132c0 con 0x55b0fef54230 2024-04-24T23:23:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.581+0000 7f34867fc640 1 -- 172.21.15.102:0/1976670931 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34780055c0 con 0x55b0fef54230 2024-04-24T23:23:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.581+0000 7f34867fc640 1 -- 172.21.15.102:0/1976670931 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478003850 con 0x55b0fef54230 2024-04-24T23:23:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.582+0000 7f34999f1740 1 -- 172.21.15.102:0/1976670931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 msgr2=0x55b0fef44440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.582+0000 7f34999f1740 1 --2- 172.21.15.102:0/1976670931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef44440 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f347800e720 tx=0x7f3478005ed0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.582+0000 7f34999f1740 1 -- 172.21.15.102:0/1976670931 shutdown_connections 2024-04-24T23:23:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.582+0000 7f34999f1740 1 --2- 172.21.15.102:0/1976670931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef54230 0x55b0fef44440 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.582+0000 7f34999f1740 1 --2- 172.21.15.102:0/1976670931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef49b60 0x55b0fef4b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.582+0000 7f34999f1740 1 --2- 172.21.15.102:0/1976670931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fef53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.582+0000 7f34999f1740 1 -- 172.21.15.102:0/1976670931 >> 172.21.15.102:0/1976670931 conn(0x55b0fef45560 msgr2=0x55b0fef43d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.582+0000 7f34999f1740 1 -- 172.21.15.102:0/1976670931 shutdown_connections 2024-04-24T23:23:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.582+0000 7f34999f1740 1 -- 172.21.15.102:0/1976670931 wait complete. 2024-04-24T23:23:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.583+0000 7f34999f1740 1 -- start start 2024-04-24T23:23:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.583+0000 7f34999f1740 1 Processor -- start 2024-04-24T23:23:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.583+0000 7f34999f1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fefc6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34999f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 0x55b0fef5bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34949a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fefc6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34949a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fefc6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:39654/0 (socket says 172.21.15.102:39654) 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34949a5640 1 -- 172.21.15.102:0/291538250 learned_addr learned my addr 172.21.15.102:0/291538250 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34999f1740 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 0x55b0fef5c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34901ab9b0 con 0x55b0fef49b60 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34901ace20 con 0x55b0fef49240 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34901ac080 con 0x55b0fef54230 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34999f1740 10 client.0 _ll_register_callbacks cb 0x7f34800b9560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.584+0000 7f34951a6640 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 0x55b0fef5c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.585+0000 7f34949a5640 1 -- 172.21.15.102:0/291538250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 msgr2=0x55b0fef5c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.585+0000 7f34949a5640 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 0x55b0fef5c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.585+0000 7f34949a5640 1 -- 172.21.15.102:0/291538250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 msgr2=0x55b0fef5bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.585+0000 7f34949a5640 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 0x55b0fef5bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.585+0000 7f34949a5640 1 -- 172.21.15.102:0/291538250 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fefc2cd0 con 0x55b0fef49240 2024-04-24T23:23:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.585+0000 7f34949a5640 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fefc6950 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b0fef67380 tx=0x7f34880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da139a8311bf230c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.585+0000 7f346ffff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:23:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.585+0000 7f346ffff640 1 -- 172.21.15.102:0/291538250 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348801b040 con 0x55b0fef49240 2024-04-24T23:23:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.585+0000 7f346ffff640 1 -- 172.21.15.102:0/291538250 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8623+0+0 (secure 0 0 0) 0x7f3488021420 con 0x55b0fef49240 2024-04-24T23:23:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.586+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34780055c0 con 0x55b0fef49240 2024-04-24T23:23:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.586+0000 7f346ffff640 1 -- 172.21.15.102:0/291538250 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3488021620 con 0x55b0fef49240 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.586+0000 7f34999f1740 10 client.25607 fetch_fsmap learned FSMap version 26 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.586+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0fefc91c0 con 0x55b0fef49240 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.586+0000 7f346ffff640 1 -- 172.21.15.102:0/291538250 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3488021c70 con 0x55b0fef49240 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.587+0000 7f346ffff640 1 -- 172.21.15.102:0/291538250 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f348802e070 con 0x55b0fef49240 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.587+0000 7f34999f1740 10 client.25607 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.587+0000 7f34999f1740 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.587+0000 7f34999f1740 20 client.25607 populate_metadata read hostname 'smithi102' 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.587+0000 7f34999f1740 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55b0fefc98f0 0x55b0fefcbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.587+0000 7f34999f1740 4 client.25607 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.587+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f34780055c0 con 0x55b0fefc98f0 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.587+0000 7f348ffff640 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55b0fefc98f0 0x55b0fefcbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.588+0000 7f346ffff640 10 client.25607 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.588+0000 7f348ffff640 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55b0fefc98f0 0x55b0fefcbd10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bfadf5c16d25b99 server_cookie=a6ba51e709c799fc in_seq=0 out_seq=0 2024-04-24T23:23:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f346ffff640 1 -- 172.21.15.102:0/291538250 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f34780055c0 con 0x55b0fefc98f0 2024-04-24T23:23:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f346ffff640 10 client.25607 handle_command_reply: tid=0 2024-04-24T23:23:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 client.25607 shutdown 2024-04-24T23:23:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 msgr2=0x55b0fefc6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fefc6950 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b0fef67380 tx=0x7f34880046c0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 shutdown_connections 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0fef54230 0x55b0fef5c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55b0fefc98f0 0x55b0fefcbd10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b0fef49b60 0x55b0fef5bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 --2- 172.21.15.102:0/291538250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0fef49240 0x55b0fefc6950 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 >> 172.21.15.102:0/291538250 conn(0x55b0fef45560 msgr2=0x55b0fef445c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f346ffff640 0 client.25607 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 shutdown_connections 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.600+0000 7f34999f1740 1 -- 172.21.15.102:0/291538250 wait complete. 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f34999f1740 20 client.25607 trim_cache size 0 max 16384 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 -- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f347c0a2e20 msgr2=0x7f347c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f347c0a2e20 0x7f347c0a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3488009990 tx=0x7f3488009110 comp rx=0 tx=0).stop 2024-04-24T23:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 -- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34901020f0 msgr2=0x7f3490198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34901020f0 0x7f3490198480 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3480006010 tx=0x7f3480003c70 comp rx=0 tx=0).stop 2024-04-24T23:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 -- 172.21.15.102:0/2539924211 shutdown_connections 2024-04-24T23:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3490107740 0x7f349019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f347c0a2e20 0x7f347c0a52e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34901020f0 0x7f3490198480 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 --2- 172.21.15.102:0/2539924211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3490101730 0x7f3490197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 -- 172.21.15.102:0/2539924211 >> 172.21.15.102:0/2539924211 conn(0x7f34900fd200 msgr2=0x7f34900fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 -- 172.21.15.102:0/2539924211 shutdown_connections 2024-04-24T23:23:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:10.601+0000 7f3496c09640 1 -- 172.21.15.102:0/2539924211 wait complete. 2024-04-24T23:23:11.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:11 smithi102 ceph-mon[29216]: pgmap v314: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 24 MiB/s wr, 6.17k op/s 2024-04-24T23:23:11.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:11 smithi161 ceph-mon[32709]: pgmap v314: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 24 MiB/s wr, 6.17k op/s 2024-04-24T23:23:11.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:11 smithi183 ceph-mon[33137]: pgmap v314: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 24 MiB/s wr, 6.17k op/s 2024-04-24T23:23:13.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:13 smithi102 ceph-mon[29216]: pgmap v315: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 24 MiB/s wr, 6.17k op/s 2024-04-24T23:23:13.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:13 smithi161 ceph-mon[32709]: pgmap v315: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 24 MiB/s wr, 6.17k op/s 2024-04-24T23:23:13.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:13 smithi183 ceph-mon[33137]: pgmap v315: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 24 MiB/s wr, 6.17k op/s 2024-04-24T23:23:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:15 smithi102 ceph-mon[29216]: pgmap v316: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 25 MiB/s wr, 6.31k op/s 2024-04-24T23:23:15.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:15 smithi102 ceph-mon[29216]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T23:23:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:15 smithi161 ceph-mon[32709]: pgmap v316: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 25 MiB/s wr, 6.31k op/s 2024-04-24T23:23:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:15 smithi161 ceph-mon[32709]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T23:23:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:15 smithi183 ceph-mon[33137]: pgmap v316: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 25 MiB/s wr, 6.31k op/s 2024-04-24T23:23:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:15 smithi183 ceph-mon[33137]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T23:23:16.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:16 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:16.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:16 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:16.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:16 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:17.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:17 smithi102 ceph-mon[29216]: pgmap v318: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 26 MiB/s wr, 6.54k op/s 2024-04-24T23:23:17.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:17 smithi161 ceph-mon[32709]: pgmap v318: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 26 MiB/s wr, 6.54k op/s 2024-04-24T23:23:17.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:17 smithi183 ceph-mon[33137]: pgmap v318: 97 pgs: 97 active+clean; 15 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 26 MiB/s wr, 6.54k op/s 2024-04-24T23:23:19.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:19 smithi102 ceph-mon[29216]: pgmap v319: 97 pgs: 97 active+clean; 15 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 27 MiB/s wr, 6.75k op/s 2024-04-24T23:23:19.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:19 smithi161 ceph-mon[32709]: pgmap v319: 97 pgs: 97 active+clean; 15 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 27 MiB/s wr, 6.75k op/s 2024-04-24T23:23:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:19 smithi183 ceph-mon[33137]: pgmap v319: 97 pgs: 97 active+clean; 15 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 27 MiB/s wr, 6.75k op/s 2024-04-24T23:23:21.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:21 smithi102 ceph-mon[29216]: pgmap v320: 97 pgs: 97 active+clean; 15 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:23:21.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:21 smithi161 ceph-mon[32709]: pgmap v320: 97 pgs: 97 active+clean; 15 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:23:21.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:21 smithi183 ceph-mon[33137]: pgmap v320: 97 pgs: 97 active+clean; 15 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:23:23.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:23 smithi102 ceph-mon[29216]: pgmap v321: 97 pgs: 97 active+clean; 15 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:23:23.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:23 smithi161 ceph-mon[32709]: pgmap v321: 97 pgs: 97 active+clean; 15 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:23:23.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:23 smithi183 ceph-mon[33137]: pgmap v321: 97 pgs: 97 active+clean; 15 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 25 MiB/s wr, 6.27k op/s 2024-04-24T23:23:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.757+0000 7f854488c640 1 -- 172.21.15.102:0/3213205146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85401097a0 msgr2=0x7f8540109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.757+0000 7f854488c640 1 --2- 172.21.15.102:0/3213205146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85401097a0 0x7f8540109b80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8534009900 tx=0x7f853402a3f0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.757+0000 7f854488c640 1 -- 172.21.15.102:0/3213205146 shutdown_connections 2024-04-24T23:23:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.757+0000 7f854488c640 1 --2- 172.21.15.102:0/3213205146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8540104150 0x7f8540110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.757+0000 7f854488c640 1 --2- 172.21.15.102:0/3213205146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8540103790 0x7f8540103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.757+0000 7f854488c640 1 --2- 172.21.15.102:0/3213205146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85401097a0 0x7f8540109b80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.757+0000 7f854488c640 1 -- 172.21.15.102:0/3213205146 >> 172.21.15.102:0/3213205146 conn(0x7f85400ff260 msgr2=0x7f8540101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.758+0000 7f854488c640 1 -- 172.21.15.102:0/3213205146 shutdown_connections 2024-04-24T23:23:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.758+0000 7f854488c640 1 -- 172.21.15.102:0/3213205146 wait complete. 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.759+0000 7f854488c640 1 Processor -- start 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.759+0000 7f854488c640 1 -- start start 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.759+0000 7f854488c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8540103790 0x7f85401a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.759+0000 7f854488c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8540104150 0x7f85401a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.759+0000 7f854488c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85401097a0 0x7f854010f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f854488c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8540076f60 con 0x7f8540104150 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f854488c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8540076de0 con 0x7f8540103790 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f854488c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85400770e0 con 0x7f85401097a0 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f853effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8540104150 0x7f85401a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f853effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8540104150 0x7f85401a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47876/0 (socket says 172.21.15.102:47876) 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f853effd640 1 -- 172.21.15.102:0/3338918140 learned_addr learned my addr 172.21.15.102:0/3338918140 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f853ffff640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85401097a0 0x7f854010f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f853effd640 1 -- 172.21.15.102:0/3338918140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85401097a0 msgr2=0x7f854010f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f853effd640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85401097a0 0x7f854010f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f853effd640 1 -- 172.21.15.102:0/3338918140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8540103790 msgr2=0x7f85401a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f853effd640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8540103790 0x7f85401a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.760+0000 7f853effd640 1 -- 172.21.15.102:0/3338918140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854010fd60 con 0x7f8540104150 2024-04-24T23:23:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.761+0000 7f853effd640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8540104150 0x7f85401a4ab0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f853000d950 tx=0x7f853000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.762+0000 7f853cff9640 1 -- 172.21.15.102:0/3338918140 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530017070 con 0x7f8540104150 2024-04-24T23:23:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.762+0000 7f853cff9640 1 -- 172.21.15.102:0/3338918140 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8530009c30 con 0x7f8540104150 2024-04-24T23:23:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.762+0000 7f853cff9640 1 -- 172.21.15.102:0/3338918140 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853000f620 con 0x7f8540104150 2024-04-24T23:23:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.762+0000 7f854488c640 1 -- 172.21.15.102:0/3338918140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8540110050 con 0x7f8540104150 2024-04-24T23:23:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.762+0000 7f854488c640 1 -- 172.21.15.102:0/3338918140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401ab310 con 0x7f8540104150 2024-04-24T23:23:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.766+0000 7f85487f9740 1 Processor -- start 2024-04-24T23:23:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.766+0000 7f85487f9740 1 -- start start 2024-04-24T23:23:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.766+0000 7f853cff9640 1 -- 172.21.15.102:0/3338918140 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f85300109a0 con 0x7f8540104150 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.766+0000 7f85487f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39ab3b0 0x55abc39ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.766+0000 7f85487f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 0x55abc39b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.766+0000 7f85487f9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39b63a0 0x55abc39b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.766+0000 7f85487f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55abc390fcb0 con 0x55abc39ab3b0 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.766+0000 7f85487f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55abc38b85b0 con 0x55abc39abcd0 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.766+0000 7f85487f9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55abc390bc70 con 0x55abc39b63a0 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39b63a0 0x55abc39b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39b63a0 0x55abc39b8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:34456/0 (socket says 172.21.15.102:34456) 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853effd640 1 -- 172.21.15.102:0/3490558619 learned_addr learned my addr 172.21.15.102:0/3490558619 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853ffff640 1 --2- 172.21.15.102:0/3490558619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39ab3b0 0x55abc39ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853f7fe640 1 --2- 172.21.15.102:0/3490558619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 0x55abc39b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853ffff640 1 -- 172.21.15.102:0/3490558619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39b63a0 msgr2=0x55abc39b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853ffff640 1 --2- 172.21.15.102:0/3490558619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39b63a0 0x55abc39b8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853ffff640 1 -- 172.21.15.102:0/3490558619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 msgr2=0x55abc39b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853ffff640 1 --2- 172.21.15.102:0/3490558619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 0x55abc39b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.767+0000 7f853ffff640 1 -- 172.21.15.102:0/3490558619 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc3624210 con 0x55abc39ab3b0 2024-04-24T23:23:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.768+0000 7f853ffff640 1 --2- 172.21.15.102:0/3490558619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39ab3b0 0x55abc39ab790 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8538009f00 tx=0x7f8538011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d005c485153e4350 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.768+0000 7f8525ffb640 1 -- 172.21.15.102:0/3490558619 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538004d20 con 0x55abc39ab3b0 2024-04-24T23:23:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.769+0000 7f8525ffb640 1 -- 172.21.15.102:0/3490558619 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8538004050 con 0x55abc39ab3b0 2024-04-24T23:23:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.769+0000 7f8525ffb640 1 -- 172.21.15.102:0/3490558619 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538005280 con 0x55abc39ab3b0 2024-04-24T23:23:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.770+0000 7f85487f9740 1 -- 172.21.15.102:0/3490558619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39ab3b0 msgr2=0x55abc39ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.770+0000 7f85487f9740 1 --2- 172.21.15.102:0/3490558619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39ab3b0 0x55abc39ab790 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8538009f00 tx=0x7f8538011810 comp rx=0 tx=0).stop 2024-04-24T23:23:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.770+0000 7f85487f9740 1 -- 172.21.15.102:0/3490558619 shutdown_connections 2024-04-24T23:23:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.770+0000 7f85487f9740 1 --2- 172.21.15.102:0/3490558619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39b63a0 0x55abc39b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.770+0000 7f85487f9740 1 --2- 172.21.15.102:0/3490558619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 0x55abc39b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.770+0000 7f85487f9740 1 --2- 172.21.15.102:0/3490558619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39ab3b0 0x55abc39ab790 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.770+0000 7f85487f9740 1 -- 172.21.15.102:0/3490558619 >> 172.21.15.102:0/3490558619 conn(0x55abc39a76d0 msgr2=0x55abc39a7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.770+0000 7f853cff9640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85280a2ec0 0x7f85280a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.771+0000 7f853ffff640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85280a2ec0 0x7f85280a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.771+0000 7f853cff9640 1 -- 172.21.15.102:0/3338918140 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f85300cf030 con 0x7f8540104150 2024-04-24T23:23:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.772+0000 7f85487f9740 1 -- 172.21.15.102:0/3490558619 shutdown_connections 2024-04-24T23:23:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.772+0000 7f85487f9740 1 -- 172.21.15.102:0/3490558619 wait complete. 2024-04-24T23:23:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.773+0000 7f85487f9740 1 -- start start 2024-04-24T23:23:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.773+0000 7f85487f9740 1 Processor -- start 2024-04-24T23:23:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.773+0000 7f853ffff640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85280a2ec0 0x7f85280a5380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f85380049d0 tx=0x7f85380096c0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.774+0000 7f85487f9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.774+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.774+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:34464/0 (socket says 172.21.15.102:34464) 2024-04-24T23:23:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.774+0000 7f85487f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39abcd0 0x55abc39b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f85487f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39b3dd0 0x55abc39d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f85487f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55abc39ca1f0 con 0x55abc39abcd0 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f85487f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55abc38cbb50 con 0x55abc39b3dd0 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f85487f9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55abc39ca370 con 0x55abc39ab3b0 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f85487f9740 10 client.0 _ll_register_callbacks cb 0x55abc3906ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f853effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39abcd0 0x55abc39b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f853effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39abcd0 0x55abc39b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47890/0 (socket says 172.21.15.102:47890) 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f853effd640 1 -- 172.21.15.102:0/1992272008 learned_addr learned my addr 172.21.15.102:0/1992272008 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f853effd640 1 -- 172.21.15.102:0/1992272008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 msgr2=0x55abc39b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f853effd640 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f853effd640 1 -- 172.21.15.102:0/1992272008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39b3dd0 msgr2=0x55abc39d3660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:23:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f853effd640 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39b3dd0 0x55abc39d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.775+0000 7f853effd640 1 -- 172.21.15.102:0/1992272008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc36f0620 con 0x55abc39abcd0 2024-04-24T23:23:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.777+0000 7f853effd640 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39abcd0 0x55abc39b3890 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8530097910 tx=0x7f85300cc740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a48a0fca36217e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.777+0000 7f851effd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:23:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.782+0000 7f851effd640 1 -- 172.21.15.102:0/1992272008 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530020040 con 0x55abc39abcd0 2024-04-24T23:23:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.783+0000 7f851effd640 1 -- 172.21.15.102:0/1992272008 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f85300a3c90 con 0x55abc39abcd0 2024-04-24T23:23:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.783+0000 7f85487f9740 1 -- 172.21.15.102:0/1992272008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8538004050 con 0x55abc39abcd0 2024-04-24T23:23:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.784+0000 7f851effd640 1 -- 172.21.15.102:0/1992272008 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f853009ae10 con 0x55abc39abcd0 2024-04-24T23:23:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.784+0000 7f85487f9740 10 client.16221 fetch_fsmap learned FSMap version 26 2024-04-24T23:23:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.784+0000 7f85487f9740 1 -- 172.21.15.102:0/1992272008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abc39d66c0 con 0x55abc39abcd0 2024-04-24T23:23:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.784+0000 7f851effd640 1 -- 172.21.15.102:0/1992272008 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8538004050 con 0x55abc39abcd0 2024-04-24T23:23:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.785+0000 7f851effd640 1 -- 172.21.15.102:0/1992272008 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f85300b4140 con 0x55abc39abcd0 2024-04-24T23:23:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.785+0000 7f85487f9740 10 client.16221 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:23:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.785+0000 7f85487f9740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:23:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.785+0000 7f85487f9740 20 client.16221 populate_metadata read hostname 'smithi102' 2024-04-24T23:23:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.785+0000 7f85487f9740 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55abc39d7b80 0x55abc39d9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.785+0000 7f85487f9740 4 client.16221 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.785+0000 7f853f7fe640 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55abc39d7b80 0x55abc39d9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.785+0000 7f85487f9740 1 -- 172.21.15.102:0/1992272008 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abc391d1f0 con 0x55abc39d7b80 2024-04-24T23:23:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.786+0000 7f853f7fe640 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55abc39d7b80 0x55abc39d9f70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59801c7e6dcf556 server_cookie=eff15122d84a2560 in_seq=0 out_seq=0 2024-04-24T23:23:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.786+0000 7f851effd640 10 client.16221 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.788+0000 7f851effd640 1 -- 172.21.15.102:0/1992272008 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abc391d1f0 con 0x55abc39d7b80 2024-04-24T23:23:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.788+0000 7f851effd640 10 client.16221 handle_command_reply: tid=0 2024-04-24T23:23:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.788+0000 7f85487f9740 1 client.16221 shutdown 2024-04-24T23:23:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.789+0000 7f85487f9740 1 -- 172.21.15.102:0/1992272008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39abcd0 msgr2=0x55abc39b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.789+0000 7f85487f9740 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39abcd0 0x55abc39b3890 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8530097910 tx=0x7f85300cc740 comp rx=0 tx=0).stop 2024-04-24T23:23:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.793+0000 7f85487f9740 1 -- 172.21.15.102:0/1992272008 shutdown_connections 2024-04-24T23:23:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.793+0000 7f85487f9740 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39b3dd0 0x55abc39d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.793+0000 7f85487f9740 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55abc39d7b80 0x55abc39d9f70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.793+0000 7f85487f9740 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39abcd0 0x55abc39b3890 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.793+0000 7f85487f9740 1 --2- 172.21.15.102:0/1992272008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.793+0000 7f85487f9740 1 -- 172.21.15.102:0/1992272008 >> 172.21.15.102:0/1992272008 conn(0x55abc39a76d0 msgr2=0x55abc39ad450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.793+0000 7f851effd640 0 client.16221 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.793+0000 7f85487f9740 1 -- 172.21.15.102:0/1992272008 shutdown_connections 2024-04-24T23:23:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.793+0000 7f85487f9740 1 -- 172.21.15.102:0/1992272008 wait complete. 2024-04-24T23:23:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.794+0000 7f85487f9740 20 client.16221 trim_cache size 0 max 16384 2024-04-24T23:23:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.826+0000 7f85487f9740 1 Processor -- start 2024-04-24T23:23:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.826+0000 7f85487f9740 1 -- start start 2024-04-24T23:23:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.827+0000 7f85487f9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39df830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.827+0000 7f85487f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 0x55abc39dfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.827+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.827+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39df830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:34478/0 (socket says 172.21.15.102:34478) 2024-04-24T23:23:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.827+0000 7f853f7fe640 1 -- 172.21.15.102:0/2514501464 learned_addr learned my addr 172.21.15.102:0/2514501464 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.827+0000 7f853effd640 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 0x55abc39dfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.827+0000 7f85487f9740 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.828+0000 7f85487f9740 1 -- 172.21.15.102:0/2514501464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85401ad960 con 0x55abc39b3dd0 2024-04-24T23:23:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.828+0000 7f85487f9740 1 -- 172.21.15.102:0/2514501464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f854011ac10 con 0x55abc39abcd0 2024-04-24T23:23:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.828+0000 7f853ffff640 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.828+0000 7f85487f9740 1 -- 172.21.15.102:0/2514501464 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f854004fd50 con 0x55abc39ab3b0 2024-04-24T23:23:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.828+0000 7f853f7fe640 1 -- 172.21.15.102:0/2514501464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 msgr2=0x55abc39dfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.828+0000 7f853f7fe640 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 0x55abc39dfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.828+0000 7f853f7fe640 1 -- 172.21.15.102:0/2514501464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 msgr2=0x55abc39d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.828+0000 7f853f7fe640 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.828+0000 7f853f7fe640 1 -- 172.21.15.102:0/2514501464 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc39a6fb0 con 0x55abc39ab3b0 2024-04-24T23:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.829+0000 7f853f7fe640 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39df830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8534009900 tx=0x7f8534032a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64586201bbb4c81b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.829+0000 7f85257fa640 1 -- 172.21.15.102:0/2514501464 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853404b070 con 0x55abc39ab3b0 2024-04-24T23:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.829+0000 7f85257fa640 1 -- 172.21.15.102:0/2514501464 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8534032bb0 con 0x55abc39ab3b0 2024-04-24T23:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.829+0000 7f85257fa640 1 -- 172.21.15.102:0/2514501464 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534035050 con 0x55abc39ab3b0 2024-04-24T23:23:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 -- 172.21.15.102:0/2514501464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 msgr2=0x55abc39df830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39df830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8534009900 tx=0x7f8534032a00 comp rx=0 tx=0).stop 2024-04-24T23:23:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 -- 172.21.15.102:0/2514501464 shutdown_connections 2024-04-24T23:23:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39abcd0 0x55abc39dfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 --2- 172.21.15.102:0/2514501464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39ab3b0 0x55abc39df830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 -- 172.21.15.102:0/2514501464 >> 172.21.15.102:0/2514501464 conn(0x55abc39a76d0 msgr2=0x55abc39b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 -- 172.21.15.102:0/2514501464 shutdown_connections 2024-04-24T23:23:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 -- 172.21.15.102:0/2514501464 wait complete. 2024-04-24T23:23:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.831+0000 7f85487f9740 1 -- start start 2024-04-24T23:23:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.832+0000 7f85487f9740 1 Processor -- start 2024-04-24T23:23:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.832+0000 7f85487f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39ab3b0 0x55abc39cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.832+0000 7f85487f9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39abcd0 0x55abc39afec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.832+0000 7f85487f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.832+0000 7f85487f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85401190d0 con 0x55abc39b3dd0 2024-04-24T23:23:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.832+0000 7f85487f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f854011a540 con 0x55abc39ab3b0 2024-04-24T23:23:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.832+0000 7f85487f9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85401197a0 con 0x55abc39abcd0 2024-04-24T23:23:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.832+0000 7f85487f9740 10 client.0 _ll_register_callbacks cb 0x7f8530097090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.833+0000 7f853ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.833+0000 7f853ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47898/0 (socket says 172.21.15.102:47898) 2024-04-24T23:23:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.833+0000 7f853ffff640 1 -- 172.21.15.102:0/3288920407 learned_addr learned my addr 172.21.15.102:0/3288920407 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.833+0000 7f853ffff640 1 -- 172.21.15.102:0/3288920407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39abcd0 msgr2=0x55abc39afec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:23:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.833+0000 7f853ffff640 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39abcd0 0x55abc39afec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.833+0000 7f853ffff640 1 -- 172.21.15.102:0/3288920407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39ab3b0 msgr2=0x55abc39cfad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:23:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.833+0000 7f853ffff640 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39ab3b0 0x55abc39cfad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.833+0000 7f853ffff640 1 -- 172.21.15.102:0/3288920407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc3a3d0a0 con 0x55abc39b3dd0 2024-04-24T23:23:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.833+0000 7f853ffff640 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39b0600 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f853801b040 tx=0x7f8538003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524bf5862899e5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.834+0000 7f851effd640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:23:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.834+0000 7f851effd640 1 -- 172.21.15.102:0/3288920407 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538036070 con 0x55abc39b3dd0 2024-04-24T23:23:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.834+0000 7f851effd640 1 -- 172.21.15.102:0/3288920407 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f85380053c0 con 0x55abc39b3dd0 2024-04-24T23:23:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.834+0000 7f85487f9740 1 -- 172.21.15.102:0/3288920407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8534032bb0 con 0x55abc39b3dd0 2024-04-24T23:23:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.834+0000 7f851effd640 1 -- 172.21.15.102:0/3288920407 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8538040050 con 0x55abc39b3dd0 2024-04-24T23:23:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.834+0000 7f85487f9740 10 client.16227 fetch_fsmap learned FSMap version 26 2024-04-24T23:23:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.834+0000 7f85487f9740 1 -- 172.21.15.102:0/3288920407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abc3a3e1c0 con 0x55abc39b3dd0 2024-04-24T23:23:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.835+0000 7f851effd640 1 -- 172.21.15.102:0/3288920407 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f853802a040 con 0x55abc39b3dd0 2024-04-24T23:23:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.835+0000 7f851effd640 1 -- 172.21.15.102:0/3288920407 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8538030ab0 con 0x55abc39b3dd0 2024-04-24T23:23:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.835+0000 7f85487f9740 10 client.16227 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:23:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.835+0000 7f85487f9740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:23:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.835+0000 7f85487f9740 20 client.16227 populate_metadata read hostname 'smithi102' 2024-04-24T23:23:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.835+0000 7f85487f9740 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55abc3a3e830 0x55abc3a40c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.836+0000 7f85487f9740 4 client.16227 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:23:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.836+0000 7f85487f9740 1 -- 172.21.15.102:0/3288920407 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8534032bb0 con 0x55abc3a3e830 2024-04-24T23:23:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.836+0000 7f853f7fe640 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55abc3a3e830 0x55abc3a40c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.837+0000 7f853f7fe640 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55abc3a3e830 0x55abc3a40c50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65801eb3e6c06abe server_cookie=6f55cd3616d1ae0e in_seq=0 out_seq=0 2024-04-24T23:23:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.837+0000 7f851effd640 10 client.16227 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f851effd640 1 -- 172.21.15.102:0/3288920407 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8534032bb0 con 0x55abc3a3e830 2024-04-24T23:23:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f851effd640 10 client.16227 handle_command_reply: tid=0 2024-04-24T23:23:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f85487f9740 1 client.16227 shutdown 2024-04-24T23:23:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f85487f9740 1 -- 172.21.15.102:0/3288920407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 msgr2=0x55abc39b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f85487f9740 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39b0600 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f853801b040 tx=0x7f8538003e00 comp rx=0 tx=0).stop 2024-04-24T23:23:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f85487f9740 1 -- 172.21.15.102:0/3288920407 shutdown_connections 2024-04-24T23:23:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f85487f9740 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55abc3a3e830 0x55abc3a40c50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f85487f9740 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abc39b3dd0 0x55abc39b0600 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f85487f9740 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abc39abcd0 0x55abc39afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f85487f9740 1 --2- 172.21.15.102:0/3288920407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abc39ab3b0 0x55abc39cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.854+0000 7f85487f9740 1 -- 172.21.15.102:0/3288920407 >> 172.21.15.102:0/3288920407 conn(0x55abc39a76d0 msgr2=0x55abc39b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.855+0000 7f851effd640 0 client.16227 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.855+0000 7f85487f9740 1 -- 172.21.15.102:0/3288920407 shutdown_connections 2024-04-24T23:23:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.855+0000 7f85487f9740 1 -- 172.21.15.102:0/3288920407 wait complete. 2024-04-24T23:23:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.855+0000 7f85487f9740 20 client.16227 trim_cache size 0 max 16384 2024-04-24T23:23:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:23:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:23:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:23:23.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:23:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.859+0000 7f854488c640 1 -- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85280a2ec0 msgr2=0x7f85280a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.859+0000 7f854488c640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85280a2ec0 0x7f85280a5380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f85380049d0 tx=0x7f85380096c0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.859+0000 7f854488c640 1 -- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8540104150 msgr2=0x7f85401a4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.859+0000 7f854488c640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8540104150 0x7f85401a4ab0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f853000d950 tx=0x7f853000de10 comp rx=0 tx=0).stop 2024-04-24T23:23:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.860+0000 7f854488c640 1 -- 172.21.15.102:0/3338918140 shutdown_connections 2024-04-24T23:23:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.860+0000 7f854488c640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85401097a0 0x7f854010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.860+0000 7f854488c640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f85280a2ec0 0x7f85280a5380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.860+0000 7f854488c640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8540104150 0x7f85401a4ab0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.860+0000 7f854488c640 1 --2- 172.21.15.102:0/3338918140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8540103790 0x7f85401a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.861+0000 7f854488c640 1 -- 172.21.15.102:0/3338918140 >> 172.21.15.102:0/3338918140 conn(0x7f85400ff260 msgr2=0x7f85400ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.862+0000 7f854488c640 1 -- 172.21.15.102:0/3338918140 shutdown_connections 2024-04-24T23:23:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:23.863+0000 7f854488c640 1 -- 172.21.15.102:0/3338918140 wait complete. 2024-04-24T23:23:25.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:24 smithi102 ceph-mon[29216]: pgmap v322: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 22 MiB/s wr, 5.49k op/s 2024-04-24T23:23:25.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:24 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:25.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:24 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:25.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:24 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:24 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:24 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:24 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:24 smithi161 ceph-mon[32709]: pgmap v322: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 22 MiB/s wr, 5.49k op/s 2024-04-24T23:23:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:24 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:24 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:24 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:24 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:24 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.110 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:24 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:24 smithi183 ceph-mon[33137]: pgmap v322: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 22 MiB/s wr, 5.49k op/s 2024-04-24T23:23:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:24 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:24 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:24 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:24 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:24 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:25.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:24 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:26.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:25 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-24T23:23:26.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:25 smithi102 ceph-mon[29216]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-24T23:23:26.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:25 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-24T23:23:26.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:25 smithi161 ceph-mon[32709]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-24T23:23:26.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:25 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-24T23:23:26.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:25 smithi183 ceph-mon[33137]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-24T23:23:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:27 smithi183 ceph-mon[33137]: pgmap v323: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 20 MiB/s wr, 5.14k op/s 2024-04-24T23:23:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:27 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:27 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:27 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:27 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:27 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:27 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:27 smithi102 ceph-mon[29216]: pgmap v323: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 20 MiB/s wr, 5.14k op/s 2024-04-24T23:23:27.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:27 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:27.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:27 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:27 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:27 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:27 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:27 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:27 smithi161 ceph-mon[32709]: pgmap v323: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 20 MiB/s wr, 5.14k op/s 2024-04-24T23:23:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:27 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:27 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:27 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:27 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:27 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:27.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:27 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:28.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:28 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-24T23:23:28.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:28 smithi183 ceph-mon[33137]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-24T23:23:28.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:28 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-24T23:23:28.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:28 smithi102 ceph-mon[29216]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-24T23:23:28.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:28 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-24T23:23:28.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:28 smithi161 ceph-mon[32709]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-24T23:23:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:29 smithi183 ceph-mon[33137]: pgmap v324: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 25 MiB/s wr, 6.32k op/s 2024-04-24T23:23:29.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:29 smithi102 ceph-mon[29216]: pgmap v324: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 25 MiB/s wr, 6.32k op/s 2024-04-24T23:23:29.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:29 smithi161 ceph-mon[32709]: pgmap v324: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 25 MiB/s wr, 6.32k op/s 2024-04-24T23:23:30.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:30 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:30.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:30 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:30.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:30 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:30.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:30 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:30.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:30 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:30.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:30 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:31.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:31 smithi102 ceph-mon[29216]: pgmap v325: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 MiB/s wr, 4.77k op/s 2024-04-24T23:23:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:31 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:31 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:31 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:31 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:31 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:31 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:31.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:31 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:31 smithi161 ceph-mon[32709]: pgmap v325: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 MiB/s wr, 4.77k op/s 2024-04-24T23:23:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:31 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:31 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:31 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:31 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:31 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:31 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:31.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:31 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:31.662 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:31 smithi183 ceph-mon[33137]: pgmap v325: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 MiB/s wr, 4.77k op/s 2024-04-24T23:23:31.662 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:31 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.662 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:31 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.662 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:31 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:31 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:23:31.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:31 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:31.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:31 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:31.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:31 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:23:32.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:32 smithi102 ceph-mon[29216]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-24T23:23:32.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:32 smithi102 ceph-mon[29216]: Unable to set osd_memory_target on smithi102 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:23:32.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:32 smithi161 ceph-mon[32709]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-24T23:23:32.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:32 smithi161 ceph-mon[32709]: Unable to set osd_memory_target on smithi102 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:23:32.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:32 smithi183 ceph-mon[33137]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-24T23:23:32.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:32 smithi183 ceph-mon[33137]: Unable to set osd_memory_target on smithi102 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-24T23:23:33.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:33 smithi102 ceph-mon[29216]: pgmap v326: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 4.28k op/s 2024-04-24T23:23:33.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:33 smithi161 ceph-mon[32709]: pgmap v326: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 4.28k op/s 2024-04-24T23:23:33.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:33 smithi183 ceph-mon[33137]: pgmap v326: 97 pgs: 97 active+clean; 15 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 4.28k op/s 2024-04-24T23:23:35.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:35 smithi102 ceph-mon[29216]: pgmap v327: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 27 MiB/s wr, 6.83k op/s 2024-04-24T23:23:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:35 smithi161 ceph-mon[32709]: pgmap v327: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 27 MiB/s wr, 6.83k op/s 2024-04-24T23:23:35.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:35 smithi183 ceph-mon[33137]: pgmap v327: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 27 MiB/s wr, 6.83k op/s 2024-04-24T23:23:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.033+0000 7f72565eb640 1 -- 172.21.15.102:0/3574560801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248094f70 msgr2=0x7f72480a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.033+0000 7f72565eb640 1 --2- 172.21.15.102:0/3574560801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248094f70 0x7f72480a1240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f724c00aa40 tx=0x7f724c02a680 comp rx=0 tx=0).stop 2024-04-24T23:23:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.033+0000 7f72565eb640 1 -- 172.21.15.102:0/3574560801 shutdown_connections 2024-04-24T23:23:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.033+0000 7f72565eb640 1 --2- 172.21.15.102:0/3574560801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248094f70 0x7f72480a1240 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.033+0000 7f72565eb640 1 --2- 172.21.15.102:0/3574560801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72480945d0 0x7f7248094a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.033+0000 7f72565eb640 1 --2- 172.21.15.102:0/3574560801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7248099340 0x7f7248099720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.033+0000 7f72565eb640 1 -- 172.21.15.102:0/3574560801 >> 172.21.15.102:0/3574560801 conn(0x7f7248009ec0 msgr2=0x7f724800a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.035+0000 7f72565eb640 1 -- 172.21.15.102:0/3574560801 shutdown_connections 2024-04-24T23:23:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.037+0000 7f72565eb640 1 -- 172.21.15.102:0/3574560801 wait complete. 2024-04-24T23:23:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.037+0000 7f72565eb640 1 Processor -- start 2024-04-24T23:23:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f72565eb640 1 -- start start 2024-04-24T23:23:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f72565eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72480945d0 0x7f724809fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f72565eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7248094f70 0x7f72480a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f72565eb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248099340 0x7f724814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f72565eb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72480a33f0 con 0x7f7248094f70 2024-04-24T23:23:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f72565eb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f72480a3270 con 0x7f7248099340 2024-04-24T23:23:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f72565eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72480a3570 con 0x7f72480945d0 2024-04-24T23:23:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f7255dea640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248099340 0x7f724814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f7255dea640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248099340 0x7f724814f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:47658/0 (socket says 172.21.15.102:47658) 2024-04-24T23:23:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f7255dea640 1 -- 172.21.15.102:0/3821515513 learned_addr learned my addr 172.21.15.102:0/3821515513 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.038+0000 7f7254de8640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7248094f70 0x7f72480a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.039+0000 7f7254de8640 1 -- 172.21.15.102:0/3821515513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72480945d0 msgr2=0x7f724809fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.039+0000 7f7254de8640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72480945d0 0x7f724809fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.039+0000 7f7254de8640 1 -- 172.21.15.102:0/3821515513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248099340 msgr2=0x7f724814f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.039+0000 7f7254de8640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248099340 0x7f724814f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.039+0000 7f7254de8640 1 -- 172.21.15.102:0/3821515513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72480a7180 con 0x7f7248094f70 2024-04-24T23:23:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.039+0000 7f7254de8640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7248094f70 0x7f72480a01e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7250050e00 tx=0x7f725006e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.039+0000 7f72467fc640 1 -- 172.21.15.102:0/3821515513 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250071630 con 0x7f7248094f70 2024-04-24T23:23:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.039+0000 7f72467fc640 1 -- 172.21.15.102:0/3821515513 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7250075070 con 0x7f7248094f70 2024-04-24T23:23:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.040+0000 7f72467fc640 1 -- 172.21.15.102:0/3821515513 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250078b90 con 0x7f7248094f70 2024-04-24T23:23:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.040+0000 7f72565eb640 1 -- 172.21.15.102:0/3821515513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72480a7470 con 0x7f7248094f70 2024-04-24T23:23:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.040+0000 7f72565eb640 1 -- 172.21.15.102:0/3821515513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72480a7940 con 0x7f7248094f70 2024-04-24T23:23:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.044+0000 7f725a63a740 1 Processor -- start 2024-04-24T23:23:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.044+0000 7f725a63a740 1 -- start start 2024-04-24T23:23:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.044+0000 7f725a63a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa500fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.044+0000 7f725a63a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa500fdcd0 0x55aa50107e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.044+0000 7f725a63a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa501083a0 0x55aa5010a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.044+0000 7f725a63a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55aa50061cb0 con 0x55aa501083a0 2024-04-24T23:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.044+0000 7f725a63a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa5000a5b0 con 0x55aa500fdcd0 2024-04-24T23:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.044+0000 7f725a63a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aa5005dc70 con 0x55aa500fd3b0 2024-04-24T23:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.045+0000 7f72555e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa500fdcd0 0x55aa50107e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.045+0000 7f72555e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa500fdcd0 0x55aa50107e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:47662/0 (socket says 172.21.15.102:47662) 2024-04-24T23:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.045+0000 7f72555e9640 1 -- 172.21.15.102:0/3319688253 learned_addr learned my addr 172.21.15.102:0/3319688253 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.045+0000 7f7255dea640 1 --2- 172.21.15.102:0/3319688253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa500fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.045+0000 7f7255dea640 1 -- 172.21.15.102:0/3319688253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa500fdcd0 msgr2=0x55aa50107e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.045+0000 7f7255dea640 1 --2- 172.21.15.102:0/3319688253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa500fdcd0 0x55aa50107e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.045+0000 7f7255dea640 1 -- 172.21.15.102:0/3319688253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa501083a0 msgr2=0x55aa5010a790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.045+0000 7f7255dea640 1 --2- 172.21.15.102:0/3319688253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa501083a0 0x55aa5010a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.045+0000 7f7255dea640 1 -- 172.21.15.102:0/3319688253 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa4fd76210 con 0x55aa500fd3b0 2024-04-24T23:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.046+0000 7f7255dea640 1 --2- 172.21.15.102:0/3319688253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa500fd790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f724c02f550 tx=0x7f724c004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2acf13ab56ed2149 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.046+0000 7f72467fc640 1 -- 172.21.15.102:0/3821515513 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f7250077030 con 0x7f7248094f70 2024-04-24T23:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.047+0000 7f72467fc640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f723c0a2e00 0x7f723c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.047+0000 7f72467fc640 1 -- 172.21.15.102:0/3821515513 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f72501322d0 con 0x7f7248094f70 2024-04-24T23:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.047+0000 7f723b7fe640 1 -- 172.21.15.102:0/3319688253 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c040070 con 0x55aa500fd3b0 2024-04-24T23:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.048+0000 7f723b7fe640 1 -- 172.21.15.102:0/3319688253 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f724c0054d0 con 0x55aa500fd3b0 2024-04-24T23:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.048+0000 7f723b7fe640 1 -- 172.21.15.102:0/3319688253 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c035040 con 0x55aa500fd3b0 2024-04-24T23:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.048+0000 7f72555e9640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f723c0a2e00 0x7f723c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.049+0000 7f725a63a740 1 -- 172.21.15.102:0/3319688253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 msgr2=0x55aa500fd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.049+0000 7f725a63a740 1 --2- 172.21.15.102:0/3319688253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa500fd790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f724c02f550 tx=0x7f724c004180 comp rx=0 tx=0).stop 2024-04-24T23:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.049+0000 7f72555e9640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f723c0a2e00 0x7f723c0a52c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55aa500fe970 tx=0x7f724000a380 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.050+0000 7f725a63a740 1 -- 172.21.15.102:0/3319688253 shutdown_connections 2024-04-24T23:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.050+0000 7f725a63a740 1 --2- 172.21.15.102:0/3319688253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa501083a0 0x55aa5010a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.050+0000 7f725a63a740 1 --2- 172.21.15.102:0/3319688253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa500fdcd0 0x55aa50107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.050+0000 7f725a63a740 1 --2- 172.21.15.102:0/3319688253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa500fd790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.050+0000 7f725a63a740 1 -- 172.21.15.102:0/3319688253 >> 172.21.15.102:0/3319688253 conn(0x55aa500f96d0 msgr2=0x55aa500f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.052+0000 7f725a63a740 1 -- 172.21.15.102:0/3319688253 shutdown_connections 2024-04-24T23:23:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.057+0000 7f725a63a740 1 -- 172.21.15.102:0/3319688253 wait complete. 2024-04-24T23:23:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.057+0000 7f725a63a740 1 -- start start 2024-04-24T23:23:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.057+0000 7f725a63a740 1 Processor -- start 2024-04-24T23:23:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.061+0000 7f725a63a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50105450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.061+0000 7f725a63a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 0x55aa50105990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.061+0000 7f725a63a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.061+0000 7f725a63a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55aa5010b130 con 0x55aa500fdcd0 2024-04-24T23:23:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.061+0000 7f725a63a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa5001db50 con 0x55aa501083a0 2024-04-24T23:23:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.061+0000 7f725a63a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aa5010b2b0 con 0x55aa500fd3b0 2024-04-24T23:23:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.061+0000 7f725a63a740 10 client.0 _ll_register_callbacks cb 0x55aa50058ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.062+0000 7f7254de8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 0x55aa50105990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.062+0000 7f7255dea640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.062+0000 7f7254de8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 0x55aa50105990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39234/0 (socket says 172.21.15.102:39234) 2024-04-24T23:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.062+0000 7f7254de8640 1 -- 172.21.15.102:0/1515897383 learned_addr learned my addr 172.21.15.102:0/1515897383 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.062+0000 7f72555e9640 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50105450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.062+0000 7f7255dea640 1 -- 172.21.15.102:0/1515897383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 msgr2=0x55aa50105450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.062+0000 7f7255dea640 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50105450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.062+0000 7f7255dea640 1 -- 172.21.15.102:0/1515897383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 msgr2=0x55aa50105990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.062+0000 7f7255dea640 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 0x55aa50105990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.063+0000 7f7255dea640 1 -- 172.21.15.102:0/1515897383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa4fe42620 con 0x55aa501083a0 2024-04-24T23:23:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.064+0000 7f7255dea640 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50105ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f724c005c40 tx=0x7f724c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea84d0a8c12e4ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.065+0000 7f7238ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:23:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.068+0000 7f7238ff9640 1 -- 172.21.15.102:0/1515897383 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c035070 con 0x55aa501083a0 2024-04-24T23:23:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.068+0000 7f7238ff9640 1 -- 172.21.15.102:0/1515897383 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f724c004e50 con 0x55aa501083a0 2024-04-24T23:23:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.068+0000 7f725a63a740 1 -- 172.21.15.102:0/1515897383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f724c0054d0 con 0x55aa501083a0 2024-04-24T23:23:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.069+0000 7f7238ff9640 1 -- 172.21.15.102:0/1515897383 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f724c040040 con 0x55aa501083a0 2024-04-24T23:23:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.070+0000 7f725a63a740 10 client.25628 fetch_fsmap learned FSMap version 26 2024-04-24T23:23:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.070+0000 7f725a63a740 1 -- 172.21.15.102:0/1515897383 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa50117490 con 0x55aa501083a0 2024-04-24T23:23:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.074+0000 7f7238ff9640 1 -- 172.21.15.102:0/1515897383 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f724c0054d0 con 0x55aa501083a0 2024-04-24T23:23:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.075+0000 7f7238ff9640 1 -- 172.21.15.102:0/1515897383 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f724c00b040 con 0x55aa501083a0 2024-04-24T23:23:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.076+0000 7f725a63a740 10 client.25628 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:23:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.076+0000 7f725a63a740 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:23:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.076+0000 7f725a63a740 20 client.25628 populate_metadata read hostname 'smithi102' 2024-04-24T23:23:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.077+0000 7f725a63a740 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55aa501187e0 0x55aa5011abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.077+0000 7f725a63a740 4 client.25628 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.077+0000 7f725a63a740 1 -- 172.21.15.102:0/1515897383 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa5006f1f0 con 0x55aa501187e0 2024-04-24T23:23:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.078+0000 7f72555e9640 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55aa501187e0 0x55aa5011abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.079+0000 7f72555e9640 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55aa501187e0 0x55aa5011abd0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492e9025fb3a6ea6 server_cookie=bd2087cd50e83917 in_seq=0 out_seq=0 2024-04-24T23:23:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.079+0000 7f7238ff9640 10 client.25628 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.080+0000 7f7238ff9640 1 -- 172.21.15.102:0/1515897383 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa5006f1f0 con 0x55aa501187e0 2024-04-24T23:23:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.080+0000 7f7238ff9640 10 client.25628 handle_command_reply: tid=0 2024-04-24T23:23:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.080+0000 7f725a63a740 1 client.25628 shutdown 2024-04-24T23:23:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.080+0000 7f725a63a740 1 -- 172.21.15.102:0/1515897383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 msgr2=0x55aa50105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.080+0000 7f725a63a740 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50105ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f724c005c40 tx=0x7f724c0045e0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.081+0000 7f725a63a740 1 -- 172.21.15.102:0/1515897383 shutdown_connections 2024-04-24T23:23:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.081+0000 7f725a63a740 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50105ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.081+0000 7f725a63a740 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55aa501187e0 0x55aa5011abd0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.081+0000 7f725a63a740 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 0x55aa50105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.081+0000 7f725a63a740 1 --2- 172.21.15.102:0/1515897383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50105450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.081+0000 7f725a63a740 1 -- 172.21.15.102:0/1515897383 >> 172.21.15.102:0/1515897383 conn(0x55aa500f96d0 msgr2=0x55aa500ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.083+0000 7f7238ff9640 0 client.25628 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.083+0000 7f725a63a740 1 -- 172.21.15.102:0/1515897383 shutdown_connections 2024-04-24T23:23:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.084+0000 7f725a63a740 1 -- 172.21.15.102:0/1515897383 wait complete. 2024-04-24T23:23:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.084+0000 7f725a63a740 20 client.25628 trim_cache size 0 max 16384 2024-04-24T23:23:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.118+0000 7f725a63a740 1 Processor -- start 2024-04-24T23:23:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.118+0000 7f725a63a740 1 -- start start 2024-04-24T23:23:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.119+0000 7f725a63a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.119+0000 7f725a63a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 0x55aa501026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.119+0000 7f72555e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.119+0000 7f72555e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:50938/0 (socket says 172.21.15.102:50938) 2024-04-24T23:23:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.119+0000 7f72555e9640 1 -- 172.21.15.102:0/444712586 learned_addr learned my addr 172.21.15.102:0/444712586 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.119+0000 7f7254de8640 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 0x55aa501026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.119+0000 7f725a63a740 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50102c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.119+0000 7f725a63a740 1 -- 172.21.15.102:0/444712586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72481639a0 con 0x55aa500fdcd0 2024-04-24T23:23:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.120+0000 7f725a63a740 1 -- 172.21.15.102:0/444712586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7248164070 con 0x55aa501083a0 2024-04-24T23:23:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.120+0000 7f7255dea640 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.120+0000 7f725a63a740 1 -- 172.21.15.102:0/444712586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7248161940 con 0x55aa500fd3b0 2024-04-24T23:23:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.120+0000 7f72555e9640 1 -- 172.21.15.102:0/444712586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 msgr2=0x55aa50102c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.120+0000 7f72555e9640 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50102c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.120+0000 7f72555e9640 1 -- 172.21.15.102:0/444712586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 msgr2=0x55aa501026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.120+0000 7f72555e9640 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 0x55aa501026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.120+0000 7f72555e9640 1 -- 172.21.15.102:0/444712586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa500f7bb0 con 0x55aa500fd3b0 2024-04-24T23:23:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.120+0000 7f72555e9640 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50107b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f724000aee0 tx=0x7f7240012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54baab8d03c01b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.121+0000 7f723affd640 1 -- 172.21.15.102:0/444712586 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7240016070 con 0x55aa500fd3b0 2024-04-24T23:23:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.121+0000 7f723affd640 1 -- 172.21.15.102:0/444712586 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f724002b070 con 0x55aa500fd3b0 2024-04-24T23:23:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.121+0000 7f723affd640 1 -- 172.21.15.102:0/444712586 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724001c420 con 0x55aa500fd3b0 2024-04-24T23:23:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.122+0000 7f725a63a740 1 -- 172.21.15.102:0/444712586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 msgr2=0x55aa50107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.122+0000 7f725a63a740 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50107b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f724000aee0 tx=0x7f7240012000 comp rx=0 tx=0).stop 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.122+0000 7f725a63a740 1 -- 172.21.15.102:0/444712586 shutdown_connections 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.122+0000 7f725a63a740 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.122+0000 7f725a63a740 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fdcd0 0x55aa501026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.122+0000 7f725a63a740 1 --2- 172.21.15.102:0/444712586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fd3b0 0x55aa50107b80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.122+0000 7f725a63a740 1 -- 172.21.15.102:0/444712586 >> 172.21.15.102:0/444712586 conn(0x55aa500f96d0 msgr2=0x55aa50104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.123+0000 7f725a63a740 1 -- 172.21.15.102:0/444712586 shutdown_connections 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.123+0000 7f725a63a740 1 -- 172.21.15.102:0/444712586 wait complete. 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.123+0000 7f725a63a740 1 -- start start 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.124+0000 7f725a63a740 1 Processor -- start 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.124+0000 7f725a63a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fd3b0 0x55aa5011ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.124+0000 7f725a63a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fdcd0 0x55aa5011b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.124+0000 7f725a63a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50176980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f7254de8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fdcd0 0x55aa5011b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f7254de8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fdcd0 0x55aa5011b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:50948/0 (socket says 172.21.15.102:50948) 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f7254de8640 1 -- 172.21.15.102:0/2060870559 learned_addr learned my addr 172.21.15.102:0/2060870559 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f7255dea640 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50176980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72480aeaa0 con 0x55aa500fd3b0 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7248161270 con 0x55aa501083a0 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72480af110 con 0x55aa500fdcd0 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f725a63a740 10 client.0 _ll_register_callbacks cb 0x7f724000a250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f7254de8640 1 -- 172.21.15.102:0/2060870559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 msgr2=0x55aa50176980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f7254de8640 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50176980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f7254de8640 1 -- 172.21.15.102:0/2060870559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fd3b0 msgr2=0x55aa5011ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f7254de8640 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fd3b0 0x55aa5011ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.125+0000 7f7254de8640 1 -- 172.21.15.102:0/2060870559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa50176ff0 con 0x55aa500fdcd0 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.126+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f724002b070 con 0x55aa500fdcd0 2024-04-24T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.126+0000 7f7254de8640 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fdcd0 0x55aa5011b350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f72500facc0 tx=0x7f72500fef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f17f0401cdc7f86e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.126+0000 7f7238ff9640 10 client.25567 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.126+0000 7f7238ff9640 1 -- 172.21.15.102:0/2060870559 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72501074d0 con 0x55aa500fdcd0 2024-04-24T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.126+0000 7f7238ff9640 1 -- 172.21.15.102:0/2060870559 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f72500faed0 con 0x55aa500fdcd0 2024-04-24T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.126+0000 7f7238ff9640 1 -- 172.21.15.102:0/2060870559 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f72500e90c0 con 0x55aa500fdcd0 2024-04-24T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.126+0000 7f725a63a740 10 client.25567 fetch_fsmap learned FSMap version 26 2024-04-24T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.126+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55aa5017e3e0 con 0x55aa500fdcd0 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.127+0000 7f7238ff9640 1 -- 172.21.15.102:0/2060870559 <== mon.1 v2:172.21.15.183:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7250081030 con 0x55aa500fdcd0 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.127+0000 7f725a63a740 10 client.25567 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.127+0000 7f725a63a740 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.127+0000 7f725a63a740 20 client.25567 populate_metadata read hostname 'smithi102' 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.127+0000 7f725a63a740 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55aa5017eab0 0x55aa50180ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.127+0000 7f725a63a740 4 client.25567 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.127+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa50069540 con 0x55aa5017eab0 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.127+0000 7f72555e9640 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55aa5017eab0 0x55aa50180ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.128+0000 7f72555e9640 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55aa5017eab0 0x55aa50180ed0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8071808735968b51 server_cookie=a17e2c553f2b0269 in_seq=0 out_seq=0 2024-04-24T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.128+0000 7f7238ff9640 10 client.25567 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f7238ff9640 1 -- 172.21.15.102:0/2060870559 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa50069540 con 0x55aa5017eab0 2024-04-24T23:23:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f7238ff9640 10 client.25567 handle_command_reply: tid=0 2024-04-24T23:23:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f725a63a740 1 client.25567 shutdown 2024-04-24T23:23:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fdcd0 msgr2=0x55aa5011b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f725a63a740 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fdcd0 0x55aa5011b350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f72500facc0 tx=0x7f72500fef60 comp rx=0 tx=0).stop 2024-04-24T23:23:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 shutdown_connections 2024-04-24T23:23:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f725a63a740 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa501083a0 0x55aa50176980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f725a63a740 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa500fdcd0 0x55aa5011b350 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f725a63a740 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55aa5017eab0 0x55aa50180ed0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f725a63a740 1 --2- 172.21.15.102:0/2060870559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa500fd3b0 0x55aa5011ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f7238ff9640 0 client.25567 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.137+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 >> 172.21.15.102:0/2060870559 conn(0x55aa500f96d0 msgr2=0x55aa500f8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.138+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 shutdown_connections 2024-04-24T23:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.139+0000 7f725a63a740 1 -- 172.21.15.102:0/2060870559 wait complete. 2024-04-24T23:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.140+0000 7f725a63a740 20 client.25567 trim_cache size 0 max 16384 2024-04-24T23:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:23:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:23:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:23:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 -- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f723c0a2e00 msgr2=0x7f723c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f723c0a2e00 0x7f723c0a52c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55aa500fe970 tx=0x7f724000a380 comp rx=0 tx=0).stop 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 -- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7248094f70 msgr2=0x7f72480a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7248094f70 0x7f72480a01e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7250050e00 tx=0x7f725006e710 comp rx=0 tx=0).stop 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 -- 172.21.15.102:0/3821515513 shutdown_connections 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248099340 0x7f724814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f723c0a2e00 0x7f723c0a52c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7248094f70 0x7f72480a01e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 --2- 172.21.15.102:0/3821515513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72480945d0 0x7f724809fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.141+0000 7f72565eb640 1 -- 172.21.15.102:0/3821515513 >> 172.21.15.102:0/3821515513 conn(0x7f7248009ec0 msgr2=0x7f72480a3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.142+0000 7f72565eb640 1 -- 172.21.15.102:0/3821515513 shutdown_connections 2024-04-24T23:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:37.142+0000 7f72565eb640 1 -- 172.21.15.102:0/3821515513 wait complete. 2024-04-24T23:23:37.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:37 smithi102 ceph-mon[29216]: pgmap v328: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2024-04-24T23:23:37.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:37 smithi161 ceph-mon[32709]: pgmap v328: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2024-04-24T23:23:37.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:37 smithi183 ceph-mon[33137]: pgmap v328: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2024-04-24T23:23:39.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:39 smithi161 ceph-mon[32709]: pgmap v329: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 MiB/s wr, 6.79k op/s 2024-04-24T23:23:39.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:39 smithi183 ceph-mon[33137]: pgmap v329: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 MiB/s wr, 6.79k op/s 2024-04-24T23:23:39.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:39 smithi102 ceph-mon[29216]: pgmap v329: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 MiB/s wr, 6.79k op/s 2024-04-24T23:23:41.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:41 smithi161 ceph-mon[32709]: pgmap v330: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 5.69k op/s 2024-04-24T23:23:41.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:41 smithi183 ceph-mon[33137]: pgmap v330: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 5.69k op/s 2024-04-24T23:23:41.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:41 smithi102 ceph-mon[29216]: pgmap v330: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 5.69k op/s 2024-04-24T23:23:43.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:43 smithi161 ceph-mon[32709]: pgmap v331: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2024-04-24T23:23:43.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:43 smithi183 ceph-mon[33137]: pgmap v331: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2024-04-24T23:23:43.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:43 smithi102 ceph-mon[29216]: pgmap v331: 97 pgs: 97 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2024-04-24T23:23:45.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:45 smithi161 ceph-mon[32709]: pgmap v332: 97 pgs: 97 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 MiB/s wr, 7.86k op/s 2024-04-24T23:23:45.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:45 smithi183 ceph-mon[33137]: pgmap v332: 97 pgs: 97 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 MiB/s wr, 7.86k op/s 2024-04-24T23:23:45.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:45 smithi102 ceph-mon[29216]: pgmap v332: 97 pgs: 97 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 MiB/s wr, 7.86k op/s 2024-04-24T23:23:47.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:47 smithi161 ceph-mon[32709]: pgmap v333: 97 pgs: 97 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 21 MiB/s wr, 5.30k op/s 2024-04-24T23:23:47.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:47 smithi183 ceph-mon[33137]: pgmap v333: 97 pgs: 97 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 21 MiB/s wr, 5.30k op/s 2024-04-24T23:23:47.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:47 smithi102 ceph-mon[29216]: pgmap v333: 97 pgs: 97 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 21 MiB/s wr, 5.30k op/s 2024-04-24T23:23:49.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:49 smithi161 ceph-mon[32709]: pgmap v334: 97 pgs: 97 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 30 MiB/s wr, 7.49k op/s 2024-04-24T23:23:49.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:49 smithi183 ceph-mon[33137]: pgmap v334: 97 pgs: 97 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 30 MiB/s wr, 7.49k op/s 2024-04-24T23:23:49.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:49 smithi102 ceph-mon[29216]: pgmap v334: 97 pgs: 97 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 30 MiB/s wr, 7.49k op/s 2024-04-24T23:23:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.300+0000 7f680c742640 1 -- 172.21.15.102:0/1526184564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680410ff30 msgr2=0x7f6804108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.300+0000 7f680c742640 1 --2- 172.21.15.102:0/1526184564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680410ff30 0x7f6804108740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6800009900 tx=0x7f680002a480 comp rx=0 tx=0).stop 2024-04-24T23:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.304+0000 7f680c742640 1 -- 172.21.15.102:0/1526184564 shutdown_connections 2024-04-24T23:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.304+0000 7f680c742640 1 --2- 172.21.15.102:0/1526184564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f680411af20 0x7f680411d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.304+0000 7f680c742640 1 --2- 172.21.15.102:0/1526184564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6804110310 0x7f6804108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.304+0000 7f680c742640 1 --2- 172.21.15.102:0/1526184564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680410ff30 0x7f6804108740 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.304+0000 7f680c742640 1 -- 172.21.15.102:0/1526184564 >> 172.21.15.102:0/1526184564 conn(0x7f680406c810 msgr2=0x7f680406cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.311+0000 7f680c742640 1 -- 172.21.15.102:0/1526184564 shutdown_connections 2024-04-24T23:23:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.313+0000 7f680c742640 1 -- 172.21.15.102:0/1526184564 wait complete. 2024-04-24T23:23:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.314+0000 7f680c742640 1 Processor -- start 2024-04-24T23:23:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.314+0000 7f680c742640 1 -- start start 2024-04-24T23:23:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.314+0000 7f680c742640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680410ff30 0x7f68041184b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.314+0000 7f680c742640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680411af20 0x7f6804111530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.314+0000 7f680c742640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6804118b00 0x7f6804111b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.314+0000 7f680c742640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6804120840 con 0x7f6804118b00 2024-04-24T23:23:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.314+0000 7f680c742640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68041206c0 con 0x7f680410ff30 2024-04-24T23:23:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.314+0000 7f680c742640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68041209c0 con 0x7f680411af20 2024-04-24T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.315+0000 7f680acdf640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6804118b00 0x7f6804111b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.315+0000 7f680acdf640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6804118b00 0x7f6804111b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41728/0 (socket says 172.21.15.102:41728) 2024-04-24T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.315+0000 7f680acdf640 1 -- 172.21.15.102:0/2325588136 learned_addr learned my addr 172.21.15.102:0/2325588136 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.316+0000 7f680acdf640 1 -- 172.21.15.102:0/2325588136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680411af20 msgr2=0x7f6804111530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.316+0000 7f680acdf640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680411af20 0x7f6804111530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.316+0000 7f680acdf640 1 -- 172.21.15.102:0/2325588136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680410ff30 msgr2=0x7f68041184b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.316+0000 7f680acdf640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680410ff30 0x7f68041184b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.316+0000 7f680acdf640 1 -- 172.21.15.102:0/2325588136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6804115fd0 con 0x7f6804118b00 2024-04-24T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.316+0000 7f680acdf640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6804118b00 0x7f6804111b10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00e9d0 tx=0x7f67fc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.317+0000 7f67fb7fe640 1 -- 172.21.15.102:0/2325588136 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc00cd80 con 0x7f6804118b00 2024-04-24T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.317+0000 7f67fb7fe640 1 -- 172.21.15.102:0/2325588136 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67fc005020 con 0x7f6804118b00 2024-04-24T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.317+0000 7f67fb7fe640 1 -- 172.21.15.102:0/2325588136 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0116d0 con 0x7f6804118b00 2024-04-24T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.317+0000 7f680c742640 1 -- 172.21.15.102:0/2325588136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68041162c0 con 0x7f6804118b00 2024-04-24T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.318+0000 7f680c742640 1 -- 172.21.15.102:0/2325588136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68041167c0 con 0x7f6804118b00 2024-04-24T23:23:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.324+0000 7f680f44d740 1 Processor -- start 2024-04-24T23:23:50.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f680f44d740 1 -- start start 2024-04-24T23:23:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f680f44d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3f5240 0x5599ad3f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f680f44d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3f5b60 0x5599ad3ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f680f44d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad400230 0x5599ad402620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f680f44d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5599ad359cb0 con 0x5599ad400230 2024-04-24T23:23:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f680f44d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5599ad3025b0 con 0x5599ad3f5240 2024-04-24T23:23:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f680f44d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599ad355c70 con 0x5599ad3f5b60 2024-04-24T23:23:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f67fb7fe640 1 -- 172.21.15.102:0/2325588136 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f67fc004b20 con 0x7f6804118b00 2024-04-24T23:23:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f6809cdd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3f5240 0x5599ad3f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f6809cdd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3f5240 0x5599ad3f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:38664/0 (socket says 172.21.15.102:38664) 2024-04-24T23:23:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f6809cdd640 1 -- 172.21.15.102:0/3985361432 learned_addr learned my addr 172.21.15.102:0/3985361432 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.325+0000 7f6809cdd640 1 --2- 172.21.15.102:0/3985361432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad400230 0x5599ad402620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.326+0000 7f6809cdd640 1 -- 172.21.15.102:0/3985361432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3f5b60 msgr2=0x5599ad3ffcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:23:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.326+0000 7f6809cdd640 1 --2- 172.21.15.102:0/3985361432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3f5b60 0x5599ad3ffcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.326+0000 7f6809cdd640 1 -- 172.21.15.102:0/3985361432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad400230 msgr2=0x5599ad402620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.326+0000 7f6809cdd640 1 --2- 172.21.15.102:0/3985361432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad400230 0x5599ad402620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.326+0000 7f6809cdd640 1 -- 172.21.15.102:0/3985361432 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ad06e210 con 0x5599ad3f5240 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.326+0000 7f67fb7fe640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f67f00a2ea0 0x7f67f00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.326+0000 7f6809cdd640 1 --2- 172.21.15.102:0/3985361432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3f5240 0x5599ad3f5620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009fb0 tx=0x7f67f40095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3108ee183e6c122e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.326+0000 7f67fb7fe640 1 -- 172.21.15.102:0/2325588136 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f67fc0cf600 con 0x7f6804118b00 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.327+0000 7f680a4de640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f67f00a2ea0 0x7f67f00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.327+0000 7f67f8ff9640 1 -- 172.21.15.102:0/3985361432 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4018020 con 0x5599ad3f5240 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.327+0000 7f67f8ff9640 1 -- 172.21.15.102:0/3985361432 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67f4004250 con 0x5599ad3f5240 2024-04-24T23:23:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.327+0000 7f67f8ff9640 1 -- 172.21.15.102:0/3985361432 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4010690 con 0x5599ad3f5240 2024-04-24T23:23:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.328+0000 7f680f44d740 1 -- 172.21.15.102:0/3985361432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3f5240 msgr2=0x5599ad3f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.328+0000 7f680f44d740 1 --2- 172.21.15.102:0/3985361432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3f5240 0x5599ad3f5620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009fb0 tx=0x7f67f40095f0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.328+0000 7f680f44d740 1 -- 172.21.15.102:0/3985361432 shutdown_connections 2024-04-24T23:23:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.328+0000 7f680f44d740 1 --2- 172.21.15.102:0/3985361432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad400230 0x5599ad402620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.328+0000 7f680f44d740 1 --2- 172.21.15.102:0/3985361432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3f5b60 0x5599ad3ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.328+0000 7f680f44d740 1 --2- 172.21.15.102:0/3985361432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3f5240 0x5599ad3f5620 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009fb0 tx=0x7f67f40095f0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.328+0000 7f680f44d740 1 -- 172.21.15.102:0/3985361432 >> 172.21.15.102:0/3985361432 conn(0x5599ad3f1560 msgr2=0x5599ad3f1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.329+0000 7f680a4de640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f67f00a2ea0 0x7f67f00a5360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6800005ec0 tx=0x7f6800005fd0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.329+0000 7f680f44d740 1 -- 172.21.15.102:0/3985361432 shutdown_connections 2024-04-24T23:23:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.329+0000 7f680f44d740 1 -- 172.21.15.102:0/3985361432 wait complete. 2024-04-24T23:23:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.329+0000 7f680f44d740 1 -- start start 2024-04-24T23:23:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.330+0000 7f680f44d740 1 Processor -- start 2024-04-24T23:23:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.330+0000 7f680f44d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3fd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.330+0000 7f680f44d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad3fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.331+0000 7f680f44d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 0x5599ad40c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.331+0000 7f6809cdd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad3fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.331+0000 7f6809cdd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad3fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:44344/0 (socket says 172.21.15.102:44344) 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.331+0000 7f6809cdd640 1 -- 172.21.15.102:0/1242019640 learned_addr learned my addr 172.21.15.102:0/1242019640 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.331+0000 7f680acdf640 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 0x5599ad40c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.331+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5599ad403020 con 0x5599ad3f5b60 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.331+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5599ad315b50 con 0x5599ad3fddb0 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.331+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599ad4031a0 con 0x5599ad400230 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.331+0000 7f680f44d740 10 client.0 _ll_register_callbacks cb 0x5599ad350ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.332+0000 7f680a4de640 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680acdf640 1 -- 172.21.15.102:0/1242019640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 msgr2=0x5599ad3fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680acdf640 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad3fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680acdf640 1 -- 172.21.15.102:0/1242019640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 msgr2=0x5599ad3fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680acdf640 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680acdf640 1 -- 172.21.15.102:0/1242019640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ad13a620 con 0x5599ad3fddb0 2024-04-24T23:23:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680a4de640 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680acdf640 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 0x5599ad40c520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0cccb0 tx=0x7f67fc004f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c842af2982ec3c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f67edffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f67edffb640 1 -- 172.21.15.102:0/1242019640 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc01f040 con 0x5599ad3fddb0 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f67edffb640 1 -- 172.21.15.102:0/1242019640 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f67fc0cf2f0 con 0x5599ad3fddb0 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f4004250 con 0x5599ad3fddb0 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f67edffb640 1 -- 172.21.15.102:0/1242019640 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67fc0cd140 con 0x5599ad3fddb0 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680f44d740 10 client.25643 fetch_fsmap learned FSMap version 26 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.333+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5599ad40f890 con 0x5599ad3fddb0 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.334+0000 7f67edffb640 1 -- 172.21.15.102:0/1242019640 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67f4004250 con 0x5599ad3fddb0 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.334+0000 7f67edffb640 1 -- 172.21.15.102:0/1242019640 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f67fc00f040 con 0x5599ad3fddb0 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.334+0000 7f680f44d740 10 client.25643 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.334+0000 7f680f44d740 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:23:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.334+0000 7f680f44d740 20 client.25643 populate_metadata read hostname 'smithi102' 2024-04-24T23:23:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.334+0000 7f680f44d740 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5599ad411260 0x5599ad4136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.334+0000 7f680f44d740 4 client.25643 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:23:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.334+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599ad3671f0 con 0x5599ad411260 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.334+0000 7f680a4de640 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5599ad411260 0x5599ad4136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.335+0000 7f680a4de640 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5599ad411260 0x5599ad4136e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7f1e62594b514ec server_cookie=7764c12811f91046 in_seq=0 out_seq=0 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.335+0000 7f67edffb640 10 client.25643 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.336+0000 7f67edffb640 1 -- 172.21.15.102:0/1242019640 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599ad3671f0 con 0x5599ad411260 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.336+0000 7f67edffb640 10 client.25643 handle_command_reply: tid=0 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.336+0000 7f680f44d740 1 client.25643 shutdown 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.336+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 msgr2=0x5599ad40c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.336+0000 7f680f44d740 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 0x5599ad40c520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0cccb0 tx=0x7f67fc004f50 comp rx=0 tx=0).stop 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 shutdown_connections 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f680f44d740 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 0x5599ad40c520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f680f44d740 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad3fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f680f44d740 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5599ad411260 0x5599ad4136e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f680f44d740 1 --2- 172.21.15.102:0/1242019640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 >> 172.21.15.102:0/1242019640 conn(0x5599ad3f1560 msgr2=0x5599ad3f7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f67edffb640 0 client.25643 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 shutdown_connections 2024-04-24T23:23:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f680f44d740 1 -- 172.21.15.102:0/1242019640 wait complete. 2024-04-24T23:23:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.337+0000 7f680f44d740 20 client.25643 trim_cache size 0 max 16384 2024-04-24T23:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.372+0000 7f680f44d740 1 Processor -- start 2024-04-24T23:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.372+0000 7f680f44d740 1 -- start start 2024-04-24T23:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.372+0000 7f680f44d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.372+0000 7f680f44d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 0x5599ad418fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.372+0000 7f680f44d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad4194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.372+0000 7f680f44d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f67e80110d0 con 0x5599ad3f5b60 2024-04-24T23:23:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680f44d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67e800e9a0 con 0x5599ad3fddb0 2024-04-24T23:23:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680a4de640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680a4de640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3f79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41758/0 (socket says 172.21.15.102:41758) 2024-04-24T23:23:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680a4de640 1 -- 172.21.15.102:0/4220926195 learned_addr learned my addr 172.21.15.102:0/4220926195 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680acdf640 1 --2- 172.21.15.102:0/4220926195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad4194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680f44d740 1 -- 172.21.15.102:0/4220926195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67e800c810 con 0x5599ad400230 2024-04-24T23:23:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680acdf640 1 -- 172.21.15.102:0/4220926195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 msgr2=0x5599ad418fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:23:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680acdf640 1 --2- 172.21.15.102:0/4220926195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 0x5599ad418fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680acdf640 1 -- 172.21.15.102:0/4220926195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 msgr2=0x5599ad3f79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680acdf640 1 --2- 172.21.15.102:0/4220926195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3f79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.373+0000 7f680acdf640 1 -- 172.21.15.102:0/4220926195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ad3fb8a0 con 0x5599ad400230 2024-04-24T23:23:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.374+0000 7f680acdf640 1 --2- 172.21.15.102:0/4220926195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad4194f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009fd0 tx=0x7f67fc09b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da36c50a1aacc4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.374+0000 7f67effff640 1 -- 172.21.15.102:0/4220926195 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0b9070 con 0x5599ad400230 2024-04-24T23:23:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.374+0000 7f67effff640 1 -- 172.21.15.102:0/4220926195 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67fc09c180 con 0x5599ad400230 2024-04-24T23:23:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.374+0000 7f67effff640 1 -- 172.21.15.102:0/4220926195 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc01f030 con 0x5599ad400230 2024-04-24T23:23:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.376+0000 7f680f44d740 1 -- 172.21.15.102:0/4220926195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 msgr2=0x5599ad4194f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.376+0000 7f680f44d740 1 --2- 172.21.15.102:0/4220926195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad4194f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009fd0 tx=0x7f67fc09b6f0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.376+0000 7f680f44d740 1 -- 172.21.15.102:0/4220926195 shutdown_connections 2024-04-24T23:23:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.376+0000 7f680f44d740 1 --2- 172.21.15.102:0/4220926195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad400230 0x5599ad4194f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.376+0000 7f680f44d740 1 --2- 172.21.15.102:0/4220926195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad3fddb0 0x5599ad418fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.376+0000 7f680f44d740 1 --2- 172.21.15.102:0/4220926195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad3f79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.376+0000 7f680f44d740 1 -- 172.21.15.102:0/4220926195 >> 172.21.15.102:0/4220926195 conn(0x5599ad3f1560 msgr2=0x5599ad3fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.376+0000 7f680f44d740 1 -- 172.21.15.102:0/4220926195 shutdown_connections 2024-04-24T23:23:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.376+0000 7f680f44d740 1 -- 172.21.15.102:0/4220926195 wait complete. 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.377+0000 7f680f44d740 1 -- start start 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.377+0000 7f680f44d740 1 Processor -- start 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.377+0000 7f680f44d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad408480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680f44d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3fddb0 0x5599ad4089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680a4de640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad408480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680a4de640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad408480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41774/0 (socket says 172.21.15.102:41774) 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680a4de640 1 -- 172.21.15.102:0/1076830793 learned_addr learned my addr 172.21.15.102:0/1076830793 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680f44d740 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad400230 0x5599ad3f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f67e800ce60 con 0x5599ad3f5b60 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67e800e2d0 con 0x5599ad400230 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67e800d530 con 0x5599ad3fddb0 2024-04-24T23:23:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f6809cdd640 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3fddb0 0x5599ad4089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680f44d740 10 client.0 _ll_register_callbacks cb 0x7f67fc097dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680acdf640 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad400230 0x5599ad3f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f6809cdd640 1 -- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad400230 msgr2=0x5599ad3f99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f6809cdd640 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad400230 0x5599ad3f99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f6809cdd640 1 -- 172.21.15.102:0/1076830793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 msgr2=0x5599ad408480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f6809cdd640 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad408480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f6809cdd640 1 -- 172.21.15.102:0/1076830793 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ad3fa200 con 0x5599ad3fddb0 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.378+0000 7f680acdf640 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad400230 0x5599ad3f99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.379+0000 7f6809cdd640 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3fddb0 0x5599ad4089c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5599ad3fcb90 tx=0x7f67f4003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a28db1ed81c59f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.379+0000 7f67edffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.379+0000 7f67edffb640 1 -- 172.21.15.102:0/1076830793 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4018050 con 0x5599ad3fddb0 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.379+0000 7f67edffb640 1 -- 172.21.15.102:0/1076830793 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f67f4003da0 con 0x5599ad3fddb0 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.379+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67fc09c180 con 0x5599ad3fddb0 2024-04-24T23:23:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.379+0000 7f67edffb640 1 -- 172.21.15.102:0/1076830793 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67f4009c00 con 0x5599ad3fddb0 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.379+0000 7f680f44d740 10 client.25579 fetch_fsmap learned FSMap version 26 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.379+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5599ad4772f0 con 0x5599ad3fddb0 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.379+0000 7f67edffb640 1 -- 172.21.15.102:0/1076830793 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67fc09c180 con 0x5599ad3fddb0 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f67edffb640 1 -- 172.21.15.102:0/1076830793 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f67f4004ab0 con 0x5599ad3fddb0 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f680f44d740 10 client.25579 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f680f44d740 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f680f44d740 20 client.25579 populate_metadata read hostname 'smithi102' 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f680f44d740 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5599ad4777b0 0x5599ad479bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f680f44d740 4 client.25579 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599ad361540 con 0x5599ad4777b0 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f680a4de640 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5599ad4777b0 0x5599ad479bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f680a4de640 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5599ad4777b0 0x5599ad479bd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5daa9ab360b39ed server_cookie=c5c8af26dda6affe in_seq=0 out_seq=0 2024-04-24T23:23:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.380+0000 7f67edffb640 10 client.25579 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.521+0000 7f67edffb640 1 -- 172.21.15.102:0/1076830793 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599ad361540 con 0x5599ad4777b0 2024-04-24T23:23:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.521+0000 7f67edffb640 10 client.25579 handle_command_reply: tid=0 2024-04-24T23:23:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.521+0000 7f680f44d740 1 client.25579 shutdown 2024-04-24T23:23:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.521+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3fddb0 msgr2=0x5599ad4089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.521+0000 7f680f44d740 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3fddb0 0x5599ad4089c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5599ad3fcb90 tx=0x7f67f4003820 comp rx=0 tx=0).stop 2024-04-24T23:23:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 shutdown_connections 2024-04-24T23:23:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f680f44d740 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599ad400230 0x5599ad3f99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f680f44d740 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ad3fddb0 0x5599ad4089c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f680f44d740 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5599ad4777b0 0x5599ad479bd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f680f44d740 1 --2- 172.21.15.102:0/1076830793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5599ad3f5b60 0x5599ad408480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 >> 172.21.15.102:0/1076830793 conn(0x5599ad3f1560 msgr2=0x5599ad3efd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f67edffb640 0 client.25579 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:23:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 shutdown_connections 2024-04-24T23:23:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f680f44d740 1 -- 172.21.15.102:0/1076830793 wait complete. 2024-04-24T23:23:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.522+0000 7f680f44d740 20 client.25579 trim_cache size 0 max 16384 2024-04-24T23:23:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:23:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:23:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:23:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:23:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 -- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f67f00a2ea0 msgr2=0x7f67f00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f67f00a2ea0 0x7f67f00a5360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6800005ec0 tx=0x7f6800005fd0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 -- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6804118b00 msgr2=0x7f6804111b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6804118b00 0x7f6804111b10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00e9d0 tx=0x7f67fc00ee90 comp rx=0 tx=0).stop 2024-04-24T23:23:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 -- 172.21.15.102:0/2325588136 shutdown_connections 2024-04-24T23:23:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f67f00a2ea0 0x7f67f00a5360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6804118b00 0x7f6804111b10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680411af20 0x7f6804111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 --2- 172.21.15.102:0/2325588136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680410ff30 0x7f68041184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:50.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.524+0000 7f680c742640 1 -- 172.21.15.102:0/2325588136 >> 172.21.15.102:0/2325588136 conn(0x7f680406c810 msgr2=0x7f680410dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:50.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.525+0000 7f680c742640 1 -- 172.21.15.102:0/2325588136 shutdown_connections 2024-04-24T23:23:50.527 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:23:50.525+0000 7f680c742640 1 -- 172.21.15.102:0/2325588136 wait complete. 2024-04-24T23:23:51.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:51 smithi102 ceph-mon[29216]: pgmap v335: 97 pgs: 97 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 24 MiB/s wr, 6.14k op/s 2024-04-24T23:23:51.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:51 smithi161 ceph-mon[32709]: pgmap v335: 97 pgs: 97 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 24 MiB/s wr, 6.14k op/s 2024-04-24T23:23:51.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:51 smithi183 ceph-mon[33137]: pgmap v335: 97 pgs: 97 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 24 MiB/s wr, 6.14k op/s 2024-04-24T23:23:53.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:53 smithi161 ceph-mon[32709]: pgmap v336: 97 pgs: 97 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:23:53.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:53 smithi183 ceph-mon[33137]: pgmap v336: 97 pgs: 97 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:23:53.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:53 smithi102 ceph-mon[29216]: pgmap v336: 97 pgs: 97 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:23:55.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:55 smithi183 ceph-mon[33137]: pgmap v337: 97 pgs: 97 active+clean; 16 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 32 MiB/s wr, 8.24k op/s 2024-04-24T23:23:55.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:55 smithi102 ceph-mon[29216]: pgmap v337: 97 pgs: 97 active+clean; 16 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 32 MiB/s wr, 8.24k op/s 2024-04-24T23:23:55.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:55 smithi161 ceph-mon[32709]: pgmap v337: 97 pgs: 97 active+clean; 16 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 32 MiB/s wr, 8.24k op/s 2024-04-24T23:23:57.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:57 smithi183 ceph-mon[33137]: pgmap v338: 97 pgs: 97 active+clean; 16 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 5.55k op/s 2024-04-24T23:23:57.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:57 smithi102 ceph-mon[29216]: pgmap v338: 97 pgs: 97 active+clean; 16 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 5.55k op/s 2024-04-24T23:23:57.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:57 smithi161 ceph-mon[32709]: pgmap v338: 97 pgs: 97 active+clean; 16 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 5.55k op/s 2024-04-24T23:23:59.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:23:59 smithi183 ceph-mon[33137]: pgmap v339: 97 pgs: 97 active+clean; 16 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 30 MiB/s wr, 7.69k op/s 2024-04-24T23:23:59.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:23:59 smithi102 ceph-mon[29216]: pgmap v339: 97 pgs: 97 active+clean; 16 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 30 MiB/s wr, 7.69k op/s 2024-04-24T23:23:59.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:23:59 smithi161 ceph-mon[32709]: pgmap v339: 97 pgs: 97 active+clean; 16 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 30 MiB/s wr, 7.69k op/s 2024-04-24T23:24:01.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:01 smithi183 ceph-mon[33137]: pgmap v340: 97 pgs: 97 active+clean; 16 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 24 MiB/s wr, 6.11k op/s 2024-04-24T23:24:01.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:01 smithi102 ceph-mon[29216]: pgmap v340: 97 pgs: 97 active+clean; 16 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 24 MiB/s wr, 6.11k op/s 2024-04-24T23:24:01.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:01 smithi161 ceph-mon[32709]: pgmap v340: 97 pgs: 97 active+clean; 16 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 24 MiB/s wr, 6.11k op/s 2024-04-24T23:24:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:03 smithi161 ceph-mon[32709]: pgmap v341: 97 pgs: 97 active+clean; 16 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 22 MiB/s wr, 5.49k op/s 2024-04-24T23:24:03.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:03 smithi183 ceph-mon[33137]: pgmap v341: 97 pgs: 97 active+clean; 16 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 22 MiB/s wr, 5.49k op/s 2024-04-24T23:24:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.765+0000 7f912ba3f640 1 -- 172.21.15.102:0/2731607384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124100520 msgr2=0x7f9124100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.765+0000 7f912ba3f640 1 --2- 172.21.15.102:0/2731607384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124100520 0x7f9124100900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9120009900 tx=0x7f912002a4d0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.768+0000 7f912ba3f640 1 -- 172.21.15.102:0/2731607384 shutdown_connections 2024-04-24T23:24:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.768+0000 7f912ba3f640 1 --2- 172.21.15.102:0/2731607384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9124105920 0x7f9124105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.768+0000 7f912ba3f640 1 --2- 172.21.15.102:0/2731607384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9124100e40 0x7f9124105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.768+0000 7f912ba3f640 1 --2- 172.21.15.102:0/2731607384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124100520 0x7f9124100900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.768+0000 7f912ba3f640 1 -- 172.21.15.102:0/2731607384 >> 172.21.15.102:0/2731607384 conn(0x7f91240fbf90 msgr2=0x7f91240fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.770+0000 7f912ba3f640 1 -- 172.21.15.102:0/2731607384 shutdown_connections 2024-04-24T23:24:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.771+0000 7f912ba3f640 1 -- 172.21.15.102:0/2731607384 wait complete. 2024-04-24T23:24:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.772+0000 7f912ba3f640 1 Processor -- start 2024-04-24T23:24:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.772+0000 7f912ba3f640 1 -- start start 2024-04-24T23:24:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.772+0000 7f912ba3f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9124100520 0x7f9124073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.772+0000 7f912ba3f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9124100e40 0x7f912406c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.772+0000 7f912ba3f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124105920 0x7f912406c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.772+0000 7f912ba3f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9124117e40 con 0x7f9124100e40 2024-04-24T23:24:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.772+0000 7f912ba3f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9124117cc0 con 0x7f9124105920 2024-04-24T23:24:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.772+0000 7f912ba3f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9124117fc0 con 0x7f9124100520 2024-04-24T23:24:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.775+0000 7f9129fdc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124105920 0x7f912406c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.775+0000 7f9129fdc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124105920 0x7f912406c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:45986/0 (socket says 172.21.15.102:45986) 2024-04-24T23:24:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.775+0000 7f9129fdc640 1 -- 172.21.15.102:0/495716301 learned_addr learned my addr 172.21.15.102:0/495716301 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.776+0000 7f9129fdc640 1 -- 172.21.15.102:0/495716301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9124100520 msgr2=0x7f9124073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.776+0000 7f9129fdc640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9124100520 0x7f9124073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.776+0000 7f9129fdc640 1 -- 172.21.15.102:0/495716301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9124100e40 msgr2=0x7f912406c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.776+0000 7f9129fdc640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9124100e40 0x7f912406c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.776+0000 7f9129fdc640 1 -- 172.21.15.102:0/495716301 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912406cd00 con 0x7f9124105920 2024-04-24T23:24:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.777+0000 7f9129fdc640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124105920 0x7f912406c650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f911c00cad0 tx=0x7f911c00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.778+0000 7f911a7fc640 1 -- 172.21.15.102:0/495716301 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c00ae20 con 0x7f9124105920 2024-04-24T23:24:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.778+0000 7f911a7fc640 1 -- 172.21.15.102:0/495716301 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f911c005020 con 0x7f9124105920 2024-04-24T23:24:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.778+0000 7f911a7fc640 1 -- 172.21.15.102:0/495716301 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c00f6b0 con 0x7f9124105920 2024-04-24T23:24:03.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.778+0000 7f912ba3f640 1 -- 172.21.15.102:0/495716301 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9124070b60 con 0x7f9124105920 2024-04-24T23:24:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.778+0000 7f912ba3f640 1 -- 172.21.15.102:0/495716301 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91240710b0 con 0x7f9124105920 2024-04-24T23:24:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.785+0000 7f911a7fc640 1 -- 172.21.15.102:0/495716301 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f911c0051d0 con 0x7f9124105920 2024-04-24T23:24:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.786+0000 7f912e82c740 1 Processor -- start 2024-04-24T23:24:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.786+0000 7f911a7fc640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f91100a2f00 0x7f91100a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.786+0000 7f911a7fc640 1 -- 172.21.15.102:0/495716301 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f911c0cf520 con 0x7f9124105920 2024-04-24T23:24:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.788+0000 7f912e82c740 1 -- start start 2024-04-24T23:24:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.788+0000 7f912e82c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d733b0 0x55efa6d73790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.788+0000 7f912e82c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d73cd0 0x55efa6d7de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.788+0000 7f912e82c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 0x55efa6d80790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.788+0000 7f912e82c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55efa6cd7cb0 con 0x55efa6d733b0 2024-04-24T23:24:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.788+0000 7f912e82c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55efa6c805b0 con 0x55efa6d73cd0 2024-04-24T23:24:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.788+0000 7f912e82c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55efa6cd3c70 con 0x55efa6d7e3a0 2024-04-24T23:24:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.789+0000 7f9128fda640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d73cd0 0x55efa6d7de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.789+0000 7f9129fdc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 0x55efa6d80790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.789+0000 7f91297db640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d733b0 0x55efa6d73790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.789+0000 7f91297db640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d733b0 0x55efa6d73790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60218/0 (socket says 172.21.15.102:60218) 2024-04-24T23:24:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.789+0000 7f91297db640 1 -- 172.21.15.102:0/3817051026 learned_addr learned my addr 172.21.15.102:0/3817051026 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.790+0000 7f91297db640 1 -- 172.21.15.102:0/3817051026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 msgr2=0x55efa6d80790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.790+0000 7f91297db640 1 --2- 172.21.15.102:0/3817051026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 0x55efa6d80790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.790+0000 7f91297db640 1 -- 172.21.15.102:0/3817051026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d73cd0 msgr2=0x55efa6d7de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.790+0000 7f91297db640 1 --2- 172.21.15.102:0/3817051026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d73cd0 0x55efa6d7de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.790+0000 7f91297db640 1 -- 172.21.15.102:0/3817051026 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efa69ec210 con 0x55efa6d733b0 2024-04-24T23:24:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.790+0000 7f9129fdc640 1 --2- 172.21.15.102:0/3817051026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 0x55efa6d80790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.790+0000 7f91297db640 1 --2- 172.21.15.102:0/3817051026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d733b0 0x55efa6d73790 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9120031540 tx=0x7f9120031570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdcd36eae0a6d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.791+0000 7f910f7fe640 1 -- 172.21.15.102:0/3817051026 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120040020 con 0x55efa6d733b0 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.791+0000 7f910f7fe640 1 -- 172.21.15.102:0/3817051026 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9120031710 con 0x55efa6d733b0 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 -- 172.21.15.102:0/3817051026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d733b0 msgr2=0x55efa6d73790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 --2- 172.21.15.102:0/3817051026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d733b0 0x55efa6d73790 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9120031540 tx=0x7f9120031570 comp rx=0 tx=0).stop 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f91297db640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f91100a2f00 0x7f91100a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 -- 172.21.15.102:0/3817051026 shutdown_connections 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 --2- 172.21.15.102:0/3817051026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 0x55efa6d80790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 --2- 172.21.15.102:0/3817051026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d73cd0 0x55efa6d7de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 --2- 172.21.15.102:0/3817051026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d733b0 0x55efa6d73790 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 -- 172.21.15.102:0/3817051026 >> 172.21.15.102:0/3817051026 conn(0x55efa6d6f6d0 msgr2=0x55efa6d6fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 -- 172.21.15.102:0/3817051026 shutdown_connections 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 -- 172.21.15.102:0/3817051026 wait complete. 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.792+0000 7f912e82c740 1 -- start start 2024-04-24T23:24:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.793+0000 7f912e82c740 1 Processor -- start 2024-04-24T23:24:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.793+0000 7f912e82c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d733b0 0x55efa6d7b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.793+0000 7f912e82c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d73cd0 0x55efa6d7b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.793+0000 7f912e82c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d7e3a0 0x55efa6d7bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.793+0000 7f912e82c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55efa6d81180 con 0x55efa6d7e3a0 2024-04-24T23:24:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.793+0000 7f912e82c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55efa6c93b50 con 0x55efa6d733b0 2024-04-24T23:24:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.793+0000 7f912e82c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55efa6d81300 con 0x55efa6d73cd0 2024-04-24T23:24:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.793+0000 7f912e82c740 10 client.0 _ll_register_callbacks cb 0x55efa6cceca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.795+0000 7f9129fdc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d7e3a0 0x55efa6d7bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.795+0000 7f9129fdc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d7e3a0 0x55efa6d7bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60228/0 (socket says 172.21.15.102:60228) 2024-04-24T23:24:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.795+0000 7f9129fdc640 1 -- 172.21.15.102:0/2295222336 learned_addr learned my addr 172.21.15.102:0/2295222336 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.795+0000 7f9128fda640 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d73cd0 0x55efa6d7b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.795+0000 7f91297db640 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d733b0 0x55efa6d7b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.796+0000 7f9128fda640 1 -- 172.21.15.102:0/2295222336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d733b0 msgr2=0x55efa6d7b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.796+0000 7f9128fda640 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d733b0 0x55efa6d7b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.796+0000 7f9128fda640 1 -- 172.21.15.102:0/2295222336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d7e3a0 msgr2=0x55efa6d7bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.796+0000 7f9128fda640 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d7e3a0 0x55efa6d7bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.796+0000 7f9128fda640 1 -- 172.21.15.102:0/2295222336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efa6ab8620 con 0x55efa6d73cd0 2024-04-24T23:24:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.797+0000 7f9128fda640 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d73cd0 0x55efa6d7b790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f912406d880 tx=0x7f9114009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77123f68c1bd20a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.797+0000 7f910cff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:24:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.797+0000 7f910cff9640 1 -- 172.21.15.102:0/2295222336 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114017070 con 0x55efa6d73cd0 2024-04-24T23:24:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.797+0000 7f910cff9640 1 -- 172.21.15.102:0/2295222336 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f9114013470 con 0x55efa6d73cd0 2024-04-24T23:24:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.798+0000 7f912e82c740 1 -- 172.21.15.102:0/2295222336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9120031710 con 0x55efa6d73cd0 2024-04-24T23:24:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.798+0000 7f91297db640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f91100a2f00 0x7f91100a53c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9120006010 tx=0x7f91200095d0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.799+0000 7f910cff9640 1 -- 172.21.15.102:0/2295222336 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9114021050 con 0x55efa6d73cd0 2024-04-24T23:24:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.799+0000 7f912e82c740 10 client.25585 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.799+0000 7f912e82c740 1 -- 172.21.15.102:0/2295222336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55efa6d8d3b0 con 0x55efa6d73cd0 2024-04-24T23:24:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.801+0000 7f910cff9640 1 -- 172.21.15.102:0/2295222336 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9120031710 con 0x55efa6d73cd0 2024-04-24T23:24:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.801+0000 7f910cff9640 1 -- 172.21.15.102:0/2295222336 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f911401c860 con 0x55efa6d73cd0 2024-04-24T23:24:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.801+0000 7f912e82c740 10 client.25585 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.801+0000 7f912e82c740 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.801+0000 7f912e82c740 20 client.25585 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.801+0000 7f912e82c740 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55efa6d8ec60 0x55efa6d910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.801+0000 7f912e82c740 4 client.25585 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.801+0000 7f912e82c740 1 -- 172.21.15.102:0/2295222336 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efa6ce51f0 con 0x55efa6d8ec60 2024-04-24T23:24:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.802+0000 7f91297db640 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55efa6d8ec60 0x55efa6d910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.805+0000 7f91297db640 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55efa6d8ec60 0x55efa6d910e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282839ebb144ed10 server_cookie=da588939ebd3c939 in_seq=0 out_seq=0 2024-04-24T23:24:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.805+0000 7f910cff9640 10 client.25585 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.807+0000 7f910cff9640 1 -- 172.21.15.102:0/2295222336 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55efa6ce51f0 con 0x55efa6d8ec60 2024-04-24T23:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.807+0000 7f910cff9640 10 client.25585 handle_command_reply: tid=0 2024-04-24T23:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.808+0000 7f912e82c740 1 client.25585 shutdown 2024-04-24T23:24:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.809+0000 7f912e82c740 1 -- 172.21.15.102:0/2295222336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d73cd0 msgr2=0x55efa6d7b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.809+0000 7f912e82c740 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d73cd0 0x55efa6d7b790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f912406d880 tx=0x7f9114009f90 comp rx=0 tx=0).stop 2024-04-24T23:24:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.812+0000 7f912e82c740 1 -- 172.21.15.102:0/2295222336 shutdown_connections 2024-04-24T23:24:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.812+0000 7f912e82c740 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55efa6d8ec60 0x55efa6d910e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.812+0000 7f912e82c740 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d7e3a0 0x55efa6d7bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.812+0000 7f912e82c740 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d73cd0 0x55efa6d7b790 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.812+0000 7f912e82c740 1 --2- 172.21.15.102:0/2295222336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d733b0 0x55efa6d7b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.812+0000 7f912e82c740 1 -- 172.21.15.102:0/2295222336 >> 172.21.15.102:0/2295222336 conn(0x55efa6d6f6d0 msgr2=0x55efa6d754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.812+0000 7f910cff9640 0 client.25585 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.813+0000 7f912e82c740 1 -- 172.21.15.102:0/2295222336 shutdown_connections 2024-04-24T23:24:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.814+0000 7f912e82c740 1 -- 172.21.15.102:0/2295222336 wait complete. 2024-04-24T23:24:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.814+0000 7f912e82c740 20 client.25585 trim_cache size 0 max 16384 2024-04-24T23:24:03.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:03 smithi102 ceph-mon[29216]: pgmap v341: 97 pgs: 97 active+clean; 16 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 22 MiB/s wr, 5.49k op/s 2024-04-24T23:24:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.846+0000 7f912e82c740 1 Processor -- start 2024-04-24T23:24:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.846+0000 7f912e82c740 1 -- start start 2024-04-24T23:24:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.848+0000 7f912e82c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d733b0 0x55efa6d8a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.848+0000 7f912e82c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d7a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.848+0000 7f912e82c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 0x55efa6d7baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.848+0000 7f912e82c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f91241c7380 con 0x55efa6d73cd0 2024-04-24T23:24:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.848+0000 7f912e82c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f912410e9c0 con 0x55efa6d733b0 2024-04-24T23:24:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.848+0000 7f912e82c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f912410c850 con 0x55efa6d7e3a0 2024-04-24T23:24:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.849+0000 7f9128fda640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d7a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.849+0000 7f9128fda640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d7a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60234/0 (socket says 172.21.15.102:60234) 2024-04-24T23:24:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.849+0000 7f9128fda640 1 -- 172.21.15.102:0/1765686 learned_addr learned my addr 172.21.15.102:0/1765686 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.850+0000 7f9128fda640 1 -- 172.21.15.102:0/1765686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 msgr2=0x55efa6d7baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.850+0000 7f9128fda640 1 --2- 172.21.15.102:0/1765686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 0x55efa6d7baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.850+0000 7f9128fda640 1 -- 172.21.15.102:0/1765686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d733b0 msgr2=0x55efa6d8a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.850+0000 7f9128fda640 1 --2- 172.21.15.102:0/1765686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d733b0 0x55efa6d8a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.850+0000 7f9128fda640 1 -- 172.21.15.102:0/1765686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efa6d72f80 con 0x55efa6d73cd0 2024-04-24T23:24:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.851+0000 7f9128fda640 1 --2- 172.21.15.102:0/1765686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d7a3b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f911400eb70 tx=0x7f911400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66f94d5756da240 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.852+0000 7f910effd640 1 -- 172.21.15.102:0/1765686 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911402a030 con 0x55efa6d73cd0 2024-04-24T23:24:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.852+0000 7f910effd640 1 -- 172.21.15.102:0/1765686 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f911401c8f0 con 0x55efa6d73cd0 2024-04-24T23:24:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.852+0000 7f910effd640 1 -- 172.21.15.102:0/1765686 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114017070 con 0x55efa6d73cd0 2024-04-24T23:24:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.853+0000 7f912e82c740 1 -- 172.21.15.102:0/1765686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 msgr2=0x55efa6d7a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.853+0000 7f912e82c740 1 --2- 172.21.15.102:0/1765686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d7a3b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f911400eb70 tx=0x7f911400eba0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.854+0000 7f912e82c740 1 -- 172.21.15.102:0/1765686 shutdown_connections 2024-04-24T23:24:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.854+0000 7f912e82c740 1 --2- 172.21.15.102:0/1765686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d7e3a0 0x55efa6d7baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.854+0000 7f912e82c740 1 --2- 172.21.15.102:0/1765686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d7a3b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.854+0000 7f912e82c740 1 --2- 172.21.15.102:0/1765686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d733b0 0x55efa6d8a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.854+0000 7f912e82c740 1 -- 172.21.15.102:0/1765686 >> 172.21.15.102:0/1765686 conn(0x55efa6d6f6d0 msgr2=0x55efa6d74c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.854+0000 7f912e82c740 1 -- 172.21.15.102:0/1765686 shutdown_connections 2024-04-24T23:24:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.855+0000 7f912e82c740 1 -- 172.21.15.102:0/1765686 wait complete. 2024-04-24T23:24:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.855+0000 7f912e82c740 1 -- start start 2024-04-24T23:24:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.855+0000 7f912e82c740 1 Processor -- start 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.856+0000 7f912e82c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d733b0 0x55efa6d81750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.856+0000 7f912e82c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d81c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.856+0000 7f912e82c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d7e3a0 0x55efa6df3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.856+0000 7f912e82c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f912410cea0 con 0x55efa6d73cd0 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.856+0000 7f912e82c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f912410e2f0 con 0x55efa6d7e3a0 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.856+0000 7f912e82c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f912410d550 con 0x55efa6d733b0 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f9129fdc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d7e3a0 0x55efa6df3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f9129fdc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d7e3a0 0x55efa6df3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:46034/0 (socket says 172.21.15.102:46034) 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f912e82c740 10 client.0 _ll_register_callbacks cb 0x7f91140092d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f9129fdc640 1 -- 172.21.15.102:0/440864071 learned_addr learned my addr 172.21.15.102:0/440864071 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f91297db640 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d733b0 0x55efa6d81750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f9128fda640 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d81c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f9128fda640 1 -- 172.21.15.102:0/440864071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d733b0 msgr2=0x55efa6d81750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f9128fda640 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d733b0 0x55efa6d81750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f9128fda640 1 -- 172.21.15.102:0/440864071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d7e3a0 msgr2=0x55efa6df3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f9128fda640 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d7e3a0 0x55efa6df3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.857+0000 7f9128fda640 1 -- 172.21.15.102:0/440864071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efa6df37e0 con 0x55efa6d73cd0 2024-04-24T23:24:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.858+0000 7f9128fda640 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d81c90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55efa6c93ca0 tx=0x7f91140098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41bea67b9076df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.858+0000 7f910cff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:24:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.859+0000 7f910cff9640 1 -- 172.21.15.102:0/440864071 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114033060 con 0x55efa6d73cd0 2024-04-24T23:24:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.859+0000 7f910cff9640 1 -- 172.21.15.102:0/440864071 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8579+0+0 (secure 0 0 0) 0x7f9114012ab0 con 0x55efa6d73cd0 2024-04-24T23:24:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.859+0000 7f912e82c740 1 -- 172.21.15.102:0/440864071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f911401c8f0 con 0x55efa6d73cd0 2024-04-24T23:24:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.860+0000 7f910cff9640 1 -- 172.21.15.102:0/440864071 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f911402a070 con 0x55efa6d73cd0 2024-04-24T23:24:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.864+0000 7f912e82c740 10 client.16266 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.864+0000 7f912e82c740 1 -- 172.21.15.102:0/440864071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55efa6df4af0 con 0x55efa6d73cd0 2024-04-24T23:24:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.873+0000 7f910cff9640 1 -- 172.21.15.102:0/440864071 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f911401c710 con 0x55efa6d73cd0 2024-04-24T23:24:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.873+0000 7f910cff9640 1 -- 172.21.15.102:0/440864071 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9114006040 con 0x55efa6d73cd0 2024-04-24T23:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.873+0000 7f912e82c740 10 client.16266 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.873+0000 7f912e82c740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.873+0000 7f912e82c740 20 client.16266 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.873+0000 7f912e82c740 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55efa6df50b0 0x55efa6df74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.874+0000 7f912e82c740 4 client.16266 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.874+0000 7f912e82c740 1 -- 172.21.15.102:0/440864071 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f911401c8f0 con 0x55efa6df50b0 2024-04-24T23:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.874+0000 7f91297db640 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55efa6df50b0 0x55efa6df74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.874+0000 7f910cff9640 10 client.16266 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:03.875+0000 7f91297db640 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55efa6df50b0 0x55efa6df74d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31698f0b4cdfb7f server_cookie=b24762bf902d4dea in_seq=0 out_seq=0 2024-04-24T23:24:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.091+0000 7f910cff9640 1 -- 172.21.15.102:0/440864071 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f911401c8f0 con 0x55efa6df50b0 2024-04-24T23:24:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.091+0000 7f910cff9640 10 client.16266 handle_command_reply: tid=0 2024-04-24T23:24:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.091+0000 7f912e82c740 1 client.16266 shutdown 2024-04-24T23:24:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.092+0000 7f912e82c740 1 -- 172.21.15.102:0/440864071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 msgr2=0x55efa6d81c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.092+0000 7f912e82c740 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d81c90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55efa6c93ca0 tx=0x7f91140098f0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.093+0000 7f912e82c740 1 -- 172.21.15.102:0/440864071 shutdown_connections 2024-04-24T23:24:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.093+0000 7f912e82c740 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efa6d7e3a0 0x55efa6df3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.093+0000 7f912e82c740 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55efa6df50b0 0x55efa6df74d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.093+0000 7f912e82c740 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efa6d73cd0 0x55efa6d81c90 secure :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55efa6c93ca0 tx=0x7f91140098f0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.093+0000 7f912e82c740 1 --2- 172.21.15.102:0/440864071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efa6d733b0 0x55efa6d81750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.093+0000 7f912e82c740 1 -- 172.21.15.102:0/440864071 >> 172.21.15.102:0/440864071 conn(0x55efa6d6f6d0 msgr2=0x55efa6d75390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.096+0000 7f910cff9640 0 client.16266 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.096+0000 7f912e82c740 1 -- 172.21.15.102:0/440864071 shutdown_connections 2024-04-24T23:24:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.096+0000 7f912e82c740 1 -- 172.21.15.102:0/440864071 wait complete. 2024-04-24T23:24:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.097+0000 7f912e82c740 20 client.16266 trim_cache size 0 max 16384 2024-04-24T23:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:24:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:24:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.103+0000 7f912ba3f640 1 -- 172.21.15.102:0/495716301 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f91100a2f00 msgr2=0x7f91100a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.104+0000 7f912ba3f640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f91100a2f00 0x7f91100a53c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9120006010 tx=0x7f91200095d0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.104+0000 7f912ba3f640 1 -- 172.21.15.102:0/495716301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124105920 msgr2=0x7f912406c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.104+0000 7f912ba3f640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124105920 0x7f912406c650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f911c00cad0 tx=0x7f911c00cf90 comp rx=0 tx=0).stop 2024-04-24T23:24:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.106+0000 7f912ba3f640 1 -- 172.21.15.102:0/495716301 shutdown_connections 2024-04-24T23:24:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.106+0000 7f912ba3f640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9124105920 0x7f912406c650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.106+0000 7f912ba3f640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f91100a2f00 0x7f91100a53c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.106+0000 7f912ba3f640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9124100e40 0x7f912406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.106+0000 7f912ba3f640 1 --2- 172.21.15.102:0/495716301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9124100520 0x7f9124073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.106+0000 7f912ba3f640 1 -- 172.21.15.102:0/495716301 >> 172.21.15.102:0/495716301 conn(0x7f91240fbf90 msgr2=0x7f91240fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.116+0000 7f912ba3f640 1 -- 172.21.15.102:0/495716301 shutdown_connections 2024-04-24T23:24:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:04.116+0000 7f912ba3f640 1 -- 172.21.15.102:0/495716301 wait complete. 2024-04-24T23:24:05.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:05 smithi183 ceph-mon[33137]: pgmap v342: 97 pgs: 97 active+clean; 16 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 29 MiB/s wr, 7.37k op/s 2024-04-24T23:24:05.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:05 smithi102 ceph-mon[29216]: pgmap v342: 97 pgs: 97 active+clean; 16 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 29 MiB/s wr, 7.37k op/s 2024-04-24T23:24:05.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:05 smithi161 ceph-mon[32709]: pgmap v342: 97 pgs: 97 active+clean; 16 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 29 MiB/s wr, 7.37k op/s 2024-04-24T23:24:07.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:07 smithi183 ceph-mon[33137]: pgmap v343: 97 pgs: 97 active+clean; 16 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 20 MiB/s wr, 5.20k op/s 2024-04-24T23:24:07.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:07 smithi183 ceph-mon[33137]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T23:24:07.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:07 smithi102 ceph-mon[29216]: pgmap v343: 97 pgs: 97 active+clean; 16 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 20 MiB/s wr, 5.20k op/s 2024-04-24T23:24:07.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:07 smithi102 ceph-mon[29216]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T23:24:07.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:07 smithi161 ceph-mon[32709]: pgmap v343: 97 pgs: 97 active+clean; 16 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 20 MiB/s wr, 5.20k op/s 2024-04-24T23:24:07.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:07 smithi161 ceph-mon[32709]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T23:24:09.765 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:09 smithi183 ceph-mon[33137]: pgmap v345: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 4.12k op/s 2024-04-24T23:24:09.827 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:09 smithi102 ceph-mon[29216]: pgmap v345: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 4.12k op/s 2024-04-24T23:24:09.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:09 smithi161 ceph-mon[32709]: pgmap v345: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 16 MiB/s wr, 4.12k op/s 2024-04-24T23:24:11.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:11 smithi183 ceph-mon[33137]: pgmap v346: 97 pgs: 9 active+clean+snaptrim_wait, 82 active+clean, 6 active+clean+snaptrim; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 23 MiB/s wr, 5.71k op/s 2024-04-24T23:24:11.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:11 smithi102 ceph-mon[29216]: pgmap v346: 97 pgs: 9 active+clean+snaptrim_wait, 82 active+clean, 6 active+clean+snaptrim; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 23 MiB/s wr, 5.71k op/s 2024-04-24T23:24:11.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:11 smithi161 ceph-mon[32709]: pgmap v346: 97 pgs: 9 active+clean+snaptrim_wait, 82 active+clean, 6 active+clean+snaptrim; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 23 MiB/s wr, 5.71k op/s 2024-04-24T23:24:13.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:13 smithi183 ceph-mon[33137]: pgmap v347: 97 pgs: 9 active+clean+snaptrim_wait, 82 active+clean, 6 active+clean+snaptrim; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 23 MiB/s wr, 5.71k op/s 2024-04-24T23:24:13.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:13 smithi102 ceph-mon[29216]: pgmap v347: 97 pgs: 9 active+clean+snaptrim_wait, 82 active+clean, 6 active+clean+snaptrim; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 23 MiB/s wr, 5.71k op/s 2024-04-24T23:24:13.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:13 smithi161 ceph-mon[32709]: pgmap v347: 97 pgs: 9 active+clean+snaptrim_wait, 82 active+clean, 6 active+clean+snaptrim; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 23 MiB/s wr, 5.71k op/s 2024-04-24T23:24:15.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:15 smithi183 ceph-mon[33137]: pgmap v348: 97 pgs: 97 active+clean; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 23 MiB/s wr, 5.72k op/s 2024-04-24T23:24:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:15 smithi183 ceph-mon[33137]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T23:24:15.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:15 smithi102 ceph-mon[29216]: pgmap v348: 97 pgs: 97 active+clean; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 23 MiB/s wr, 5.72k op/s 2024-04-24T23:24:15.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:15 smithi102 ceph-mon[29216]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T23:24:15.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:15 smithi161 ceph-mon[32709]: pgmap v348: 97 pgs: 97 active+clean; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 23 MiB/s wr, 5.72k op/s 2024-04-24T23:24:15.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:15 smithi161 ceph-mon[32709]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T23:24:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.299+0000 7fa55bfff640 1 -- 172.21.15.102:0/423832628 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554037070 con 0x7fa5641088a0 2024-04-24T23:24:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.300+0000 7fa56ba38640 1 -- 172.21.15.102:0/423832628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5641088a0 msgr2=0x7fa56410c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.300+0000 7fa56ba38640 1 --2- 172.21.15.102:0/423832628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5641088a0 0x7fa56410c810 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa554009860 tx=0x7fa55402a360 comp rx=0 tx=0).stop 2024-04-24T23:24:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.300+0000 7fa56ba38640 1 -- 172.21.15.102:0/423832628 shutdown_connections 2024-04-24T23:24:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.300+0000 7fa56ba38640 1 --2- 172.21.15.102:0/423832628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa56410cef0 0x7fa564114ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.300+0000 7fa56ba38640 1 --2- 172.21.15.102:0/423832628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5641088a0 0x7fa56410c810 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.300+0000 7fa56ba38640 1 --2- 172.21.15.102:0/423832628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa564107ef0 0x7fa5641082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.300+0000 7fa56ba38640 1 -- 172.21.15.102:0/423832628 >> 172.21.15.102:0/423832628 conn(0x7fa5640774f0 msgr2=0x7fa564106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.301+0000 7fa56ba38640 1 -- 172.21.15.102:0/423832628 shutdown_connections 2024-04-24T23:24:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.301+0000 7fa56ba38640 1 -- 172.21.15.102:0/423832628 wait complete. 2024-04-24T23:24:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.302+0000 7fa56ba38640 1 Processor -- start 2024-04-24T23:24:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.302+0000 7fa56ba38640 1 -- start start 2024-04-24T23:24:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.303+0000 7fa56ba38640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa564107ef0 0x7fa564114770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.303+0000 7fa56ba38640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56410cef0 0x7fa564110850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.303+0000 7fa56ba38640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa564110d90 0x7fa564111240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.303+0000 7fa56ba38640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa56406da50 con 0x7fa564110d90 2024-04-24T23:24:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.303+0000 7fa56ba38640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa56406d8d0 con 0x7fa56410cef0 2024-04-24T23:24:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.303+0000 7fa56ba38640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa56406dbd0 con 0x7fa564107ef0 2024-04-24T23:24:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.303+0000 7fa5697d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa564107ef0 0x7fa564114770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.303+0000 7fa5697d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa564107ef0 0x7fa564114770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:41800/0 (socket says 172.21.15.102:41800) 2024-04-24T23:24:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.303+0000 7fa5697d4640 1 -- 172.21.15.102:0/4134973364 learned_addr learned my addr 172.21.15.102:0/4134973364 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.304+0000 7fa5697d4640 1 -- 172.21.15.102:0/4134973364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56410cef0 msgr2=0x7fa564110850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:24:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.304+0000 7fa5697d4640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56410cef0 0x7fa564110850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.304+0000 7fa5697d4640 1 -- 172.21.15.102:0/4134973364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa564110d90 msgr2=0x7fa564111240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.304+0000 7fa5697d4640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa564110d90 0x7fa564111240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.305+0000 7fa5697d4640 1 -- 172.21.15.102:0/4134973364 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564081fb0 con 0x7fa564107ef0 2024-04-24T23:24:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.305+0000 7fa5697d4640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa564107ef0 0x7fa564114770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00ba50 tx=0x7fa55c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.306+0000 7fa55a7fc640 1 -- 172.21.15.102:0/4134973364 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c019070 con 0x7fa564107ef0 2024-04-24T23:24:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.306+0000 7fa55a7fc640 1 -- 172.21.15.102:0/4134973364 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa55c00cbe0 con 0x7fa564107ef0 2024-04-24T23:24:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.306+0000 7fa56ba38640 1 -- 172.21.15.102:0/4134973364 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5640822a0 con 0x7fa564107ef0 2024-04-24T23:24:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.306+0000 7fa56ba38640 1 -- 172.21.15.102:0/4134973364 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5641ab510 con 0x7fa564107ef0 2024-04-24T23:24:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.306+0000 7fa55a7fc640 1 -- 172.21.15.102:0/4134973364 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0075d0 con 0x7fa564107ef0 2024-04-24T23:24:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.315+0000 7fa56e822740 1 Processor -- start 2024-04-24T23:24:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.317+0000 7fa56e822740 1 -- start start 2024-04-24T23:24:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.317+0000 7fa56e822740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f125743b0 0x555f12574790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.317+0000 7fa56e822740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f1257ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.317+0000 7fa56e822740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f12581790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.317+0000 7fa56e822740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555f124d8cb0 con 0x555f125743b0 2024-04-24T23:24:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.317+0000 7fa56e822740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555f124815b0 con 0x555f1257f3a0 2024-04-24T23:24:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.317+0000 7fa56e822740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f124d4c70 con 0x555f12574cd0 2024-04-24T23:24:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.317+0000 7fa55a7fc640 1 -- 172.21.15.102:0/4134973364 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa55c01c070 con 0x7fa564107ef0 2024-04-24T23:24:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa568fd3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f12581790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa568fd3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f12581790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:39796/0 (socket says 172.21.15.102:39796) 2024-04-24T23:24:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa568fd3640 1 -- 172.21.15.102:0/2153517960 learned_addr learned my addr 172.21.15.102:0/2153517960 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa5697d4640 1 --2- 172.21.15.102:0/2153517960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f1257ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa568fd3640 1 -- 172.21.15.102:0/2153517960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 msgr2=0x555f1257ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa568fd3640 1 --2- 172.21.15.102:0/2153517960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f1257ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa568fd3640 1 -- 172.21.15.102:0/2153517960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f125743b0 msgr2=0x555f12574790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa568fd3640 1 --2- 172.21.15.102:0/2153517960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f125743b0 0x555f12574790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa568fd3640 1 -- 172.21.15.102:0/2153517960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f121ed210 con 0x555f1257f3a0 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.318+0000 7fa568fd3640 1 --2- 172.21.15.102:0/2153517960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f12581790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa554000c00 tx=0x7fa554009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79534dccb31dceb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.319+0000 7fa55a7fc640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa5500a2ec0 0x7fa5500a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.319+0000 7fa54f7fe640 1 -- 172.21.15.102:0/2153517960 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554037070 con 0x555f1257f3a0 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.319+0000 7fa54f7fe640 1 -- 172.21.15.102:0/2153517960 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa554038040 con 0x555f1257f3a0 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.319+0000 7fa54f7fe640 1 -- 172.21.15.102:0/2153517960 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554004b90 con 0x555f1257f3a0 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.319+0000 7fa55a7fc640 1 -- 172.21.15.102:0/4134973364 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7fa55c0ce9f0 con 0x7fa564107ef0 2024-04-24T23:24:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.319+0000 7fa569fd5640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa5500a2ec0 0x7fa5500a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 -- 172.21.15.102:0/2153517960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 msgr2=0x555f12581790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 --2- 172.21.15.102:0/2153517960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f12581790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa554000c00 tx=0x7fa554009450 comp rx=0 tx=0).stop 2024-04-24T23:24:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa569fd5640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa5500a2ec0 0x7fa5500a5380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa560005fd0 tx=0x7fa560005a20 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 -- 172.21.15.102:0/2153517960 shutdown_connections 2024-04-24T23:24:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 --2- 172.21.15.102:0/2153517960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f12581790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 --2- 172.21.15.102:0/2153517960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f1257ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 --2- 172.21.15.102:0/2153517960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f125743b0 0x555f12574790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 -- 172.21.15.102:0/2153517960 >> 172.21.15.102:0/2153517960 conn(0x555f125706d0 msgr2=0x555f12570ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 -- 172.21.15.102:0/2153517960 shutdown_connections 2024-04-24T23:24:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 -- 172.21.15.102:0/2153517960 wait complete. 2024-04-24T23:24:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 -- start start 2024-04-24T23:24:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.320+0000 7fa56e822740 1 Processor -- start 2024-04-24T23:24:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa56e822740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 0x555f1257c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa56e822740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa56e822740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f1257f3a0 0x555f1257cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa56e822740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555f12582180 con 0x555f12574cd0 2024-04-24T23:24:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa56e822740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555f12494b50 con 0x555f125743b0 2024-04-24T23:24:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa56e822740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f12582300 con 0x555f1257f3a0 2024-04-24T23:24:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa56e822740 10 client.0 _ll_register_callbacks cb 0x555f124cfca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa568fd3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa568fd3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49804/0 (socket says 172.21.15.102:49804) 2024-04-24T23:24:17.326 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.321+0000 7fa568fd3640 1 -- 172.21.15.102:0/945142278 learned_addr learned my addr 172.21.15.102:0/945142278 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.323+0000 7fa568fd3640 1 -- 172.21.15.102:0/945142278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f1257f3a0 msgr2=0x555f1257cd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:24:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.323+0000 7fa568fd3640 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f1257f3a0 0x555f1257cd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.323+0000 7fa568fd3640 1 -- 172.21.15.102:0/945142278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 msgr2=0x555f1257c2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:24:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.323+0000 7fa568fd3640 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 0x555f1257c2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.323+0000 7fa568fd3640 1 -- 172.21.15.102:0/945142278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f122b9620 con 0x555f12574cd0 2024-04-24T23:24:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.324+0000 7fa568fd3640 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257c800 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa55402a3a0 tx=0x7fa5540052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39aceef02a6d318 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.324+0000 7fa54cff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:24:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.324+0000 7fa54cff9640 1 -- 172.21.15.102:0/945142278 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554037070 con 0x555f12574cd0 2024-04-24T23:24:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.324+0000 7fa54cff9640 1 -- 172.21.15.102:0/945142278 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7fa554004b10 con 0x555f12574cd0 2024-04-24T23:24:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.324+0000 7fa56e822740 1 -- 172.21.15.102:0/945142278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa554038040 con 0x555f12574cd0 2024-04-24T23:24:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.325+0000 7fa54cff9640 1 -- 172.21.15.102:0/945142278 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa554053050 con 0x555f12574cd0 2024-04-24T23:24:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.325+0000 7fa56e822740 10 client.16272 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.325+0000 7fa56e822740 1 -- 172.21.15.102:0/945142278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555f1258e360 con 0x555f12574cd0 2024-04-24T23:24:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.326+0000 7fa54cff9640 1 -- 172.21.15.102:0/945142278 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa554038040 con 0x555f12574cd0 2024-04-24T23:24:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.326+0000 7fa54cff9640 1 -- 172.21.15.102:0/945142278 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa5540324e0 con 0x555f12574cd0 2024-04-24T23:24:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.326+0000 7fa56e822740 10 client.16272 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.326+0000 7fa56e822740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.326+0000 7fa56e822740 20 client.16272 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.326+0000 7fa56e822740 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555f1258f9e0 0x555f12591e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.326+0000 7fa56e822740 4 client.16272 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.326+0000 7fa56e822740 1 -- 172.21.15.102:0/945142278 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f124e61f0 con 0x555f1258f9e0 2024-04-24T23:24:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.327+0000 7fa5697d4640 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555f1258f9e0 0x555f12591e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.328+0000 7fa54cff9640 10 client.16272 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.329+0000 7fa5697d4640 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555f1258f9e0 0x555f12591e00 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5786821821754350 server_cookie=8553ee93fd5b0716 in_seq=0 out_seq=0 2024-04-24T23:24:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa54cff9640 1 -- 172.21.15.102:0/945142278 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f124e61f0 con 0x555f1258f9e0 2024-04-24T23:24:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa54cff9640 10 client.16272 handle_command_reply: tid=0 2024-04-24T23:24:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa56e822740 1 client.16272 shutdown 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa56e822740 1 -- 172.21.15.102:0/945142278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 msgr2=0x555f1257c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa56e822740 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257c800 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa55402a3a0 tx=0x7fa5540052f0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa56e822740 1 -- 172.21.15.102:0/945142278 shutdown_connections 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa56e822740 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f1257f3a0 0x555f1257cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa56e822740 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555f1258f9e0 0x555f12591e00 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa56e822740 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257c800 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa56e822740 1 --2- 172.21.15.102:0/945142278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 0x555f1257c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa56e822740 1 -- 172.21.15.102:0/945142278 >> 172.21.15.102:0/945142278 conn(0x555f125706d0 msgr2=0x555f125764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.330+0000 7fa54cff9640 0 client.16272 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.331+0000 7fa56e822740 1 -- 172.21.15.102:0/945142278 shutdown_connections 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.331+0000 7fa56e822740 1 -- 172.21.15.102:0/945142278 wait complete. 2024-04-24T23:24:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.331+0000 7fa56e822740 20 client.16272 trim_cache size 0 max 16384 2024-04-24T23:24:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.366+0000 7fa56e822740 1 Processor -- start 2024-04-24T23:24:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.367+0000 7fa56e822740 1 -- start start 2024-04-24T23:24:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.367+0000 7fa56e822740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f125743b0 0x555f1258b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.367+0000 7fa56e822740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa56e822740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f1257cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa5697d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f125743b0 0x555f1258b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa56e822740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641b61c0 con 0x555f12574cd0 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa5697d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f125743b0 0x555f1258b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:41828/0 (socket says 172.21.15.102:41828) 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa56e822740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641b3a90 con 0x555f1257f3a0 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa56e822740 1 -- 172.21.15.102:0/2575048524 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641b1900 con 0x555f125743b0 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa5697d4640 1 -- 172.21.15.102:0/2575048524 learned_addr learned my addr 172.21.15.102:0/2575048524 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa568fd3640 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa569fd5640 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f1257cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa5697d4640 1 -- 172.21.15.102:0/2575048524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 msgr2=0x555f1257cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa5697d4640 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f1257cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa5697d4640 1 -- 172.21.15.102:0/2575048524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 msgr2=0x555f1257b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa5697d4640 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa5697d4640 1 -- 172.21.15.102:0/2575048524 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f1256b940 con 0x555f125743b0 2024-04-24T23:24:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.368+0000 7fa568fd3640 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.369+0000 7fa5697d4640 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f125743b0 0x555f1258b090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa55c09ac80 tx=0x7fa55c0d0e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a5e4aab1f9073e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.370+0000 7fa54effd640 1 -- 172.21.15.102:0/2575048524 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c09ad50 con 0x555f125743b0 2024-04-24T23:24:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.370+0000 7fa54effd640 1 -- 172.21.15.102:0/2575048524 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa55c09b680 con 0x555f125743b0 2024-04-24T23:24:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.370+0000 7fa54effd640 1 -- 172.21.15.102:0/2575048524 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c09a8e0 con 0x555f125743b0 2024-04-24T23:24:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.372+0000 7fa56e822740 1 -- 172.21.15.102:0/2575048524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f125743b0 msgr2=0x555f1258b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.372+0000 7fa56e822740 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f125743b0 0x555f1258b090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa55c09ac80 tx=0x7fa55c0d0e40 comp rx=0 tx=0).stop 2024-04-24T23:24:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.373+0000 7fa56e822740 1 -- 172.21.15.102:0/2575048524 shutdown_connections 2024-04-24T23:24:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.373+0000 7fa56e822740 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f1257f3a0 0x555f1257cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.373+0000 7fa56e822740 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f12574cd0 0x555f1257b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.373+0000 7fa56e822740 1 --2- 172.21.15.102:0/2575048524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f125743b0 0x555f1258b090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.373+0000 7fa56e822740 1 -- 172.21.15.102:0/2575048524 >> 172.21.15.102:0/2575048524 conn(0x555f125706d0 msgr2=0x555f1256efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.373+0000 7fa56e822740 1 -- 172.21.15.102:0/2575048524 shutdown_connections 2024-04-24T23:24:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.373+0000 7fa56e822740 1 -- 172.21.15.102:0/2575048524 wait complete. 2024-04-24T23:24:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.374+0000 7fa56e822740 1 -- start start 2024-04-24T23:24:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.374+0000 7fa56e822740 1 Processor -- start 2024-04-24T23:24:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa56e822740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 0x555f12582750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa56e822740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f12582c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa5697d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 0x555f12582750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa5697d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 0x555f12582750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:39822/0 (socket says 172.21.15.102:39822) 2024-04-24T23:24:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa56e822740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f1257f3a0 0x555f125f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa56e822740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641b1f50 con 0x555f1257f3a0 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa56e822740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641b33c0 con 0x555f125743b0 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa568fd3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f12582c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa56e822740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641b2620 con 0x555f12574cd0 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa568fd3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f12582c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:41838/0 (socket says 172.21.15.102:41838) 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa568fd3640 1 -- 172.21.15.102:0/3484170812 learned_addr learned my addr 172.21.15.102:0/3484170812 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa56e822740 10 client.0 _ll_register_callbacks cb 0x7fa55c0d44e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.375+0000 7fa569fd5640 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f1257f3a0 0x555f125f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.376+0000 7fa568fd3640 1 -- 172.21.15.102:0/3484170812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 msgr2=0x555f12582750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.376+0000 7fa568fd3640 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 0x555f12582750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.376+0000 7fa568fd3640 1 -- 172.21.15.102:0/3484170812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f1257f3a0 msgr2=0x555f125f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.376+0000 7fa568fd3640 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f1257f3a0 0x555f125f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.376+0000 7fa568fd3640 1 -- 172.21.15.102:0/3484170812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f125f4470 con 0x555f12574cd0 2024-04-24T23:24:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.376+0000 7fa568fd3640 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f12582c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x555f1257bb20 tx=0x7fa5540045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=196b4d26417f5c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.376+0000 7fa54cff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.377+0000 7fa54cff9640 1 -- 172.21.15.102:0/3484170812 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55405c060 con 0x555f12574cd0 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.377+0000 7fa54cff9640 1 -- 172.21.15.102:0/3484170812 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7fa5540048c0 con 0x555f12574cd0 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.377+0000 7fa56e822740 1 -- 172.21.15.102:0/3484170812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa55c09b680 con 0x555f12574cd0 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.377+0000 7fa54cff9640 1 -- 172.21.15.102:0/3484170812 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa554037070 con 0x555f12574cd0 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.377+0000 7fa56e822740 10 client.25603 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.378+0000 7fa56e822740 1 -- 172.21.15.102:0/3484170812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555f125f56c0 con 0x555f12574cd0 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.378+0000 7fa54cff9640 1 -- 172.21.15.102:0/3484170812 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa554038040 con 0x555f12574cd0 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.378+0000 7fa54cff9640 1 -- 172.21.15.102:0/3484170812 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa554003af0 con 0x555f12574cd0 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.378+0000 7fa56e822740 10 client.25603 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.378+0000 7fa56e822740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.378+0000 7fa56e822740 20 client.25603 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.378+0000 7fa56e822740 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555f125f5e40 0x555f125f8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.379+0000 7fa56e822740 4 client.25603 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:24:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.380+0000 7fa56e822740 1 -- 172.21.15.102:0/3484170812 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa55c09b680 con 0x555f125f5e40 2024-04-24T23:24:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.380+0000 7fa5697d4640 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555f125f5e40 0x555f125f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.380+0000 7fa5697d4640 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555f125f5e40 0x555f125f8260 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc35994154ed9fae server_cookie=2b7dcd667279830a in_seq=0 out_seq=0 2024-04-24T23:24:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.381+0000 7fa54cff9640 10 client.25603 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.395+0000 7fa54cff9640 1 -- 172.21.15.102:0/3484170812 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa55c09b680 con 0x555f125f5e40 2024-04-24T23:24:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.395+0000 7fa54cff9640 10 client.25603 handle_command_reply: tid=0 2024-04-24T23:24:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.395+0000 7fa56e822740 1 client.25603 shutdown 2024-04-24T23:24:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 -- 172.21.15.102:0/3484170812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 msgr2=0x555f12582c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f12582c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x555f1257bb20 tx=0x7fa5540045a0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 -- 172.21.15.102:0/3484170812 shutdown_connections 2024-04-24T23:24:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555f125f5e40 0x555f125f8260 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f1257f3a0 0x555f125f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f12574cd0 0x555f12582c90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 --2- 172.21.15.102:0/3484170812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f125743b0 0x555f12582750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 -- 172.21.15.102:0/3484170812 >> 172.21.15.102:0/3484170812 conn(0x555f125706d0 msgr2=0x555f1256f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa54cff9640 0 client.25603 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 -- 172.21.15.102:0/3484170812 shutdown_connections 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.396+0000 7fa56e822740 1 -- 172.21.15.102:0/3484170812 wait complete. 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.397+0000 7fa56e822740 20 client.25603 trim_cache size 0 max 16384 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:24:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:24:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.401+0000 7fa56ba38640 1 -- 172.21.15.102:0/4134973364 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa5500a2ec0 msgr2=0x7fa5500a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.401+0000 7fa56ba38640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa5500a2ec0 0x7fa5500a5380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa560005fd0 tx=0x7fa560005a20 comp rx=0 tx=0).stop 2024-04-24T23:24:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.401+0000 7fa56ba38640 1 -- 172.21.15.102:0/4134973364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa564107ef0 msgr2=0x7fa564114770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.401+0000 7fa56ba38640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa564107ef0 0x7fa564114770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00ba50 tx=0x7fa55c00bf10 comp rx=0 tx=0).stop 2024-04-24T23:24:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.404+0000 7fa56ba38640 1 -- 172.21.15.102:0/4134973364 shutdown_connections 2024-04-24T23:24:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.404+0000 7fa56ba38640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa5500a2ec0 0x7fa5500a5380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.404+0000 7fa56ba38640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa564110d90 0x7fa564111240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.404+0000 7fa56ba38640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56410cef0 0x7fa564110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.404+0000 7fa56ba38640 1 --2- 172.21.15.102:0/4134973364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa564107ef0 0x7fa564114770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.404+0000 7fa56ba38640 1 -- 172.21.15.102:0/4134973364 >> 172.21.15.102:0/4134973364 conn(0x7fa5640774f0 msgr2=0x7fa56410e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.405+0000 7fa56ba38640 1 -- 172.21.15.102:0/4134973364 shutdown_connections 2024-04-24T23:24:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:17.405+0000 7fa56ba38640 1 -- 172.21.15.102:0/4134973364 wait complete. 2024-04-24T23:24:17.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:17 smithi183 ceph-mon[33137]: pgmap v350: 97 pgs: 97 active+clean; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 24 MiB/s wr, 6.13k op/s 2024-04-24T23:24:17.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:17 smithi102 ceph-mon[29216]: pgmap v350: 97 pgs: 97 active+clean; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 24 MiB/s wr, 6.13k op/s 2024-04-24T23:24:17.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:17 smithi161 ceph-mon[32709]: pgmap v350: 97 pgs: 97 active+clean; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 24 MiB/s wr, 6.13k op/s 2024-04-24T23:24:19.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:19 smithi183 ceph-mon[33137]: pgmap v351: 97 pgs: 97 active+clean; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 5.76k op/s 2024-04-24T23:24:19.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:19 smithi102 ceph-mon[29216]: pgmap v351: 97 pgs: 97 active+clean; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 5.76k op/s 2024-04-24T23:24:19.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:19 smithi161 ceph-mon[32709]: pgmap v351: 97 pgs: 97 active+clean; 16 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 5.76k op/s 2024-04-24T23:24:21.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:21 smithi183 ceph-mon[33137]: pgmap v352: 97 pgs: 97 active+clean; 16 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 23 MiB/s wr, 5.86k op/s 2024-04-24T23:24:21.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:21 smithi102 ceph-mon[29216]: pgmap v352: 97 pgs: 97 active+clean; 16 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 23 MiB/s wr, 5.86k op/s 2024-04-24T23:24:21.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:21 smithi161 ceph-mon[32709]: pgmap v352: 97 pgs: 97 active+clean; 16 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 23 MiB/s wr, 5.86k op/s 2024-04-24T23:24:23.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:23 smithi183 ceph-mon[33137]: pgmap v353: 97 pgs: 97 active+clean; 16 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 23 MiB/s wr, 5.87k op/s 2024-04-24T23:24:23.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:23 smithi102 ceph-mon[29216]: pgmap v353: 97 pgs: 97 active+clean; 16 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 23 MiB/s wr, 5.87k op/s 2024-04-24T23:24:23.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:23 smithi161 ceph-mon[32709]: pgmap v353: 97 pgs: 97 active+clean; 16 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 23 MiB/s wr, 5.87k op/s 2024-04-24T23:24:25.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:25 smithi183 ceph-mon[33137]: pgmap v354: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 5.76k op/s 2024-04-24T23:24:25.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:25 smithi102 ceph-mon[29216]: pgmap v354: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 5.76k op/s 2024-04-24T23:24:25.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:25 smithi161 ceph-mon[32709]: pgmap v354: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 5.76k op/s 2024-04-24T23:24:27.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:27 smithi183 ceph-mon[33137]: pgmap v355: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 21 MiB/s wr, 5.39k op/s 2024-04-24T23:24:27.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:27 smithi102 ceph-mon[29216]: pgmap v355: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 21 MiB/s wr, 5.39k op/s 2024-04-24T23:24:27.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:27 smithi161 ceph-mon[32709]: pgmap v355: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 21 MiB/s wr, 5.39k op/s 2024-04-24T23:24:29.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:29 smithi183 ceph-mon[33137]: pgmap v356: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 21 MiB/s wr, 5.23k op/s 2024-04-24T23:24:29.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:29 smithi102 ceph-mon[29216]: pgmap v356: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 21 MiB/s wr, 5.23k op/s 2024-04-24T23:24:29.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:29 smithi161 ceph-mon[32709]: pgmap v356: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 21 MiB/s wr, 5.23k op/s 2024-04-24T23:24:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.597+0000 7fda01fa9640 1 -- 172.21.15.102:0/1860955430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc10dc60 msgr2=0x7fd9fc10e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.597+0000 7fda01fa9640 1 --2- 172.21.15.102:0/1860955430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc10dc60 0x7fd9fc10e040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400afd0 tx=0x7fd9f402b430 comp rx=0 tx=0).stop 2024-04-24T23:24:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.597+0000 7fda01fa9640 1 -- 172.21.15.102:0/1860955430 shutdown_connections 2024-04-24T23:24:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.597+0000 7fda01fa9640 1 --2- 172.21.15.102:0/1860955430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9fc10d0a0 0x7fd9fc079a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.597+0000 7fda01fa9640 1 --2- 172.21.15.102:0/1860955430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9fc10e610 0x7fd9fc10c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.597+0000 7fda01fa9640 1 --2- 172.21.15.102:0/1860955430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc10dc60 0x7fd9fc10e040 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400afd0 tx=0x7fd9f402b430 comp rx=0 tx=0).stop 2024-04-24T23:24:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.597+0000 7fda01fa9640 1 -- 172.21.15.102:0/1860955430 >> 172.21.15.102:0/1860955430 conn(0x7fd9fc06c810 msgr2=0x7fd9fc06cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.598+0000 7fda01fa9640 1 -- 172.21.15.102:0/1860955430 shutdown_connections 2024-04-24T23:24:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.598+0000 7fda01fa9640 1 -- 172.21.15.102:0/1860955430 wait complete. 2024-04-24T23:24:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.598+0000 7fda01fa9640 1 Processor -- start 2024-04-24T23:24:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.599+0000 7fda01fa9640 1 -- start start 2024-04-24T23:24:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.606+0000 7fda01fa9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9fc10d0a0 0x7fd9fc083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.606+0000 7fda01fa9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9fc07d480 0x7fd9fc07d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.606+0000 7fda01fa9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc07de20 0x7fd9fc1ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.606+0000 7fda01fa9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc07bbb0 con 0x7fd9fc10d0a0 2024-04-24T23:24:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.606+0000 7fda01fa9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc07ba30 con 0x7fd9fc07de20 2024-04-24T23:24:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.606+0000 7fda01fa9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc07bd30 con 0x7fd9fc07d480 2024-04-24T23:24:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.607+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc07de20 0x7fd9fc1ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.607+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc07de20 0x7fd9fc1ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:58650/0 (socket says 172.21.15.102:58650) 2024-04-24T23:24:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.607+0000 7fd9fbfff640 1 -- 172.21.15.102:0/3789312123 learned_addr learned my addr 172.21.15.102:0/3789312123 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.607+0000 7fd9fbfff640 1 -- 172.21.15.102:0/3789312123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9fc07d480 msgr2=0x7fd9fc07d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.607+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9fc07d480 0x7fd9fc07d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.607+0000 7fd9fbfff640 1 -- 172.21.15.102:0/3789312123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9fc10d0a0 msgr2=0x7fd9fc083410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:24:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.607+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9fc10d0a0 0x7fd9fc083410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.607+0000 7fd9fbfff640 1 -- 172.21.15.102:0/3789312123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc1cd370 con 0x7fd9fc07de20 2024-04-24T23:24:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.608+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc07de20 0x7fd9fc1ccd60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009fd0 tx=0x7fd9e400b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.608+0000 7fd9f8ff9640 1 -- 172.21.15.102:0/3789312123 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4011830 con 0x7fd9fc07de20 2024-04-24T23:24:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.608+0000 7fda01fa9640 1 -- 172.21.15.102:0/3789312123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc1cd660 con 0x7fd9fc07de20 2024-04-24T23:24:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.608+0000 7fda01fa9640 1 -- 172.21.15.102:0/3789312123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc1cdbb0 con 0x7fd9fc07de20 2024-04-24T23:24:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.608+0000 7fd9f8ff9640 1 -- 172.21.15.102:0/3789312123 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9e4016070 con 0x7fd9fc07de20 2024-04-24T23:24:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.608+0000 7fd9f8ff9640 1 -- 172.21.15.102:0/3789312123 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4010e50 con 0x7fd9fc07de20 2024-04-24T23:24:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.612+0000 7fd9f8ff9640 1 -- 172.21.15.102:0/3789312123 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd9e401b4a0 con 0x7fd9fc07de20 2024-04-24T23:24:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.613+0000 7fd9f8ff9640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd9e80a3090 0x7fd9e80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.613+0000 7fd9f8ff9640 1 -- 172.21.15.102:0/3789312123 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7fd9e40cfa20 con 0x7fd9fc07de20 2024-04-24T23:24:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.615+0000 7fda04d96740 1 Processor -- start 2024-04-24T23:24:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.615+0000 7fda04d96740 1 -- start start 2024-04-24T23:24:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.615+0000 7fda04d96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f65620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.615+0000 7fda04d96740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.615+0000 7fda04d96740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f70230 0x5621e7f72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.615+0000 7fda04d96740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5621e7ec9cb0 con 0x5621e7f65b60 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.615+0000 7fda04d96740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5621e7e725b0 con 0x5621e7f70230 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.615+0000 7fda04d96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621e7ec5c70 con 0x5621e7f65240 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.616+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f70230 0x5621e7f72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.616+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f70230 0x5621e7f72620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:58666/0 (socket says 172.21.15.102:58666) 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.616+0000 7fd9fbfff640 1 -- 172.21.15.102:0/1937120296 learned_addr learned my addr 172.21.15.102:0/1937120296 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.616+0000 7fd9fbfff640 1 -- 172.21.15.102:0/1937120296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 msgr2=0x5621e7f65620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.616+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/1937120296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f65620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.616+0000 7fd9fbfff640 1 -- 172.21.15.102:0/1937120296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 msgr2=0x5621e7f6fcf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.616+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/1937120296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6fcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.616+0000 7fd9fbfff640 1 -- 172.21.15.102:0/1937120296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e7bde210 con 0x5621e7f70230 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.617+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/1937120296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f70230 0x5621e7f72620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4010980 tx=0x7fd9e400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38815cb807d7b272 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.617+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd9e80a3090 0x7fd9e80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.618+0000 7fd9f1ffb640 1 -- 172.21.15.102:0/1937120296 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e40bc070 con 0x5621e7f70230 2024-04-24T23:24:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.618+0000 7fd9f1ffb640 1 -- 172.21.15.102:0/1937120296 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9e40a4c20 con 0x5621e7f70230 2024-04-24T23:24:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.619+0000 7fda04d96740 1 -- 172.21.15.102:0/1937120296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f70230 msgr2=0x5621e7f72620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.619+0000 7fda04d96740 1 --2- 172.21.15.102:0/1937120296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f70230 0x5621e7f72620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4010980 tx=0x7fd9e400bf90 comp rx=0 tx=0).stop 2024-04-24T23:24:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.619+0000 7fda04d96740 1 -- 172.21.15.102:0/1937120296 shutdown_connections 2024-04-24T23:24:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.619+0000 7fda04d96740 1 --2- 172.21.15.102:0/1937120296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f70230 0x5621e7f72620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.619+0000 7fda04d96740 1 --2- 172.21.15.102:0/1937120296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.619+0000 7fda04d96740 1 --2- 172.21.15.102:0/1937120296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f65620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.619+0000 7fda04d96740 1 -- 172.21.15.102:0/1937120296 >> 172.21.15.102:0/1937120296 conn(0x5621e7f61560 msgr2=0x5621e7f61940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.620+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd9e80a3090 0x7fd9e80a5550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd9f402ba20 tx=0x7fd9f4000f40 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.626+0000 7fda04d96740 1 -- 172.21.15.102:0/1937120296 shutdown_connections 2024-04-24T23:24:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.626+0000 7fda04d96740 1 -- 172.21.15.102:0/1937120296 wait complete. 2024-04-24T23:24:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.626+0000 7fda04d96740 1 -- start start 2024-04-24T23:24:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fda04d96740 1 Processor -- start 2024-04-24T23:24:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fda04d96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f72540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fda04d96740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fda04d96740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f70220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fda04d96740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5621e7f73010 con 0x5621e7f65b60 2024-04-24T23:24:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fda04d96740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5621e7e85b50 con 0x5621e7f6fdc0 2024-04-24T23:24:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fda04d96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621e7f73190 con 0x5621e7f65240 2024-04-24T23:24:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fda04d96740 10 client.0 _ll_register_callbacks cb 0x5621e7ec0ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f70220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fd9fbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f70220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:58682/0 (socket says 172.21.15.102:58682) 2024-04-24T23:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fd9fbfff640 1 -- 172.21.15.102:0/263715654 learned_addr learned my addr 172.21.15.102:0/263715654 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.627+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f72540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.628+0000 7fd9fbfff640 1 -- 172.21.15.102:0/263715654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 msgr2=0x5621e7f72540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.628+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f72540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.628+0000 7fd9fbfff640 1 -- 172.21.15.102:0/263715654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 msgr2=0x5621e7f6f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.628+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.628+0000 7fd9fbfff640 1 -- 172.21.15.102:0/263715654 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e7caa620 con 0x5621e7f6fdc0 2024-04-24T23:24:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.632+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f70220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4098410 tx=0x7fd9e40a0490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2bb4d6f22c78752 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.633+0000 7fd9daffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.635+0000 7fd9daffd640 1 -- 172.21.15.102:0/263715654 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e40cd550 con 0x5621e7f6fdc0 2024-04-24T23:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.635+0000 7fda04d96740 1 -- 172.21.15.102:0/263715654 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621e7ed71f0 con 0x5621e7f6fdc0 2024-04-24T23:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.635+0000 7fd9daffd640 1 -- 172.21.15.102:0/263715654 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7fd9e409b400 con 0x5621e7f6fdc0 2024-04-24T23:24:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.635+0000 7fd9daffd640 1 -- 172.21.15.102:0/263715654 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9e40a4c20 con 0x5621e7f6fdc0 2024-04-24T23:24:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.635+0000 7fda04d96740 10 client.25682 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.635+0000 7fda04d96740 1 -- 172.21.15.102:0/263715654 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621e7f73010 con 0x5621e7f6fdc0 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.636+0000 7fd9daffd640 1 -- 172.21.15.102:0/263715654 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5621e7ed71f0 con 0x5621e7f6fdc0 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.638+0000 7fd9daffd640 1 -- 172.21.15.102:0/263715654 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd9e409af90 con 0x5621e7f6fdc0 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.638+0000 7fda04d96740 10 client.25682 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.638+0000 7fda04d96740 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.638+0000 7fda04d96740 20 client.25682 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.638+0000 7fda04d96740 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5621e7f80a70 0x5621e7f82ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.638+0000 7fda04d96740 4 client.25682 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.638+0000 7fda04d96740 1 -- 172.21.15.102:0/263715654 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621e7ed1540 con 0x5621e7f80a70 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.639+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5621e7f80a70 0x5621e7f82ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.640+0000 7fd9daffd640 10 client.25682 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.642+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5621e7f80a70 0x5621e7f82ed0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85beffa266081f8f server_cookie=a97de95a6cd17eb5 in_seq=0 out_seq=0 2024-04-24T23:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fd9daffd640 1 -- 172.21.15.102:0/263715654 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621e7ed1540 con 0x5621e7f80a70 2024-04-24T23:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fd9daffd640 10 client.25682 handle_command_reply: tid=0 2024-04-24T23:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fda04d96740 1 client.25682 shutdown 2024-04-24T23:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fda04d96740 1 -- 172.21.15.102:0/263715654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 msgr2=0x5621e7f70220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fda04d96740 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f70220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4098410 tx=0x7fd9e40a0490 comp rx=0 tx=0).stop 2024-04-24T23:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fda04d96740 1 -- 172.21.15.102:0/263715654 shutdown_connections 2024-04-24T23:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fda04d96740 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f70220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fda04d96740 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5621e7f80a70 0x5621e7f82ed0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fda04d96740 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fda04d96740 1 --2- 172.21.15.102:0/263715654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f72540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fda04d96740 1 -- 172.21.15.102:0/263715654 >> 172.21.15.102:0/263715654 conn(0x5621e7f61560 msgr2=0x5621e7f67260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.644+0000 7fd9daffd640 0 client.25682 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.646+0000 7fda04d96740 1 -- 172.21.15.102:0/263715654 shutdown_connections 2024-04-24T23:24:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.649+0000 7fda04d96740 1 -- 172.21.15.102:0/263715654 wait complete. 2024-04-24T23:24:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.649+0000 7fda04d96740 20 client.25682 trim_cache size 0 max 16384 2024-04-24T23:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.684+0000 7fda04d96740 1 Processor -- start 2024-04-24T23:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.685+0000 7fda04d96740 1 -- start start 2024-04-24T23:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.685+0000 7fda04d96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f6f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fda04d96740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f65b60 0x5621e7f72070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fda04d96740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f6fdc0 0x5621e7f887e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f6f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f6f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:57934/0 (socket says 172.21.15.102:57934) 2024-04-24T23:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fd9faffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f65b60 0x5621e7f72070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fd9faffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f65b60 0x5621e7f72070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:58698/0 (socket says 172.21.15.102:58698) 2024-04-24T23:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fd9fb7fe640 1 -- 172.21.15.102:0/531115762 learned_addr learned my addr 172.21.15.102:0/531115762 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fda04d96740 1 -- 172.21.15.102:0/531115762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc011150 con 0x5621e7f6fdc0 2024-04-24T23:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fda04d96740 1 -- 172.21.15.102:0/531115762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc00ea20 con 0x5621e7f65b60 2024-04-24T23:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.686+0000 7fda04d96740 1 -- 172.21.15.102:0/531115762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc00c890 con 0x5621e7f65240 2024-04-24T23:24:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.689+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/531115762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f6fdc0 0x5621e7f887e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.692+0000 7fd9fb7fe640 1 -- 172.21.15.102:0/531115762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f65b60 msgr2=0x5621e7f72070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.692+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/531115762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f65b60 0x5621e7f72070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.692+0000 7fd9fb7fe640 1 -- 172.21.15.102:0/531115762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f6fdc0 msgr2=0x5621e7f887e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.692+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/531115762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f6fdc0 0x5621e7f887e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.692+0000 7fd9fb7fe640 1 -- 172.21.15.102:0/531115762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e7f72b40 con 0x5621e7f65240 2024-04-24T23:24:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.693+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/531115762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f6f7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4035040 tx=0x7fd9f4039000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981499c200a51f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.693+0000 7fd9f17fa640 1 -- 172.21.15.102:0/531115762 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4054070 con 0x5621e7f65240 2024-04-24T23:24:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.693+0000 7fd9f17fa640 1 -- 172.21.15.102:0/531115762 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9f4003b70 con 0x5621e7f65240 2024-04-24T23:24:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.693+0000 7fd9f17fa640 1 -- 172.21.15.102:0/531115762 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f404fe60 con 0x5621e7f65240 2024-04-24T23:24:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.694+0000 7fda04d96740 1 -- 172.21.15.102:0/531115762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 msgr2=0x5621e7f6f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.694+0000 7fda04d96740 1 --2- 172.21.15.102:0/531115762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f6f7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4035040 tx=0x7fd9f4039000 comp rx=0 tx=0).stop 2024-04-24T23:24:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.695+0000 7fda04d96740 1 -- 172.21.15.102:0/531115762 shutdown_connections 2024-04-24T23:24:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.695+0000 7fda04d96740 1 --2- 172.21.15.102:0/531115762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f6fdc0 0x5621e7f887e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.695+0000 7fda04d96740 1 --2- 172.21.15.102:0/531115762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f65b60 0x5621e7f72070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.695+0000 7fda04d96740 1 --2- 172.21.15.102:0/531115762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f6f7d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.695+0000 7fda04d96740 1 -- 172.21.15.102:0/531115762 >> 172.21.15.102:0/531115762 conn(0x5621e7f61560 msgr2=0x5621e7f6f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.697+0000 7fda04d96740 1 -- 172.21.15.102:0/531115762 shutdown_connections 2024-04-24T23:24:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.697+0000 7fda04d96740 1 -- 172.21.15.102:0/531115762 wait complete. 2024-04-24T23:24:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.697+0000 7fda04d96740 1 -- start start 2024-04-24T23:24:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.698+0000 7fda04d96740 1 Processor -- start 2024-04-24T23:24:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.698+0000 7fda04d96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f78900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.699+0000 7fda04d96740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.699+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f78900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.699+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f78900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:57936/0 (socket says 172.21.15.102:57936) 2024-04-24T23:24:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.699+0000 7fd9fb7fe640 1 -- 172.21.15.102:0/3679209151 learned_addr learned my addr 172.21.15.102:0/3679209151 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.699+0000 7fda04d96740 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.699+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc00cee0 con 0x5621e7f65b60 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.699+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc00e350 con 0x5621e7f6fdc0 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.699+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.700+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc00d5b0 con 0x5621e7f65240 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.700+0000 7fd9faffd640 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.700+0000 7fda04d96740 10 client.0 _ll_register_callbacks cb 0x7fd9e409b930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.700+0000 7fd9fbfff640 1 -- 172.21.15.102:0/3679209151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 msgr2=0x5621e7f78900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.700+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f78900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.700+0000 7fd9fbfff640 1 -- 172.21.15.102:0/3679209151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 msgr2=0x5621e7f6cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.700+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.700+0000 7fd9fbfff640 1 -- 172.21.15.102:0/3679209151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e7f6bc20 con 0x5621e7f6fdc0 2024-04-24T23:24:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.701+0000 7fd9fbfff640 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f6d430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4010980 tx=0x7fd9e40047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4da4b12e38471789 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.701+0000 7fd9daffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.701+0000 7fd9daffd640 1 -- 172.21.15.102:0/3679209151 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e409b400 con 0x5621e7f6fdc0 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.701+0000 7fd9daffd640 1 -- 172.21.15.102:0/3679209151 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7fd9e409b5a0 con 0x5621e7f6fdc0 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.701+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9f4003b70 con 0x5621e7f6fdc0 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.702+0000 7fd9daffd640 1 -- 172.21.15.102:0/3679209151 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9e4097d00 con 0x5621e7f6fdc0 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.702+0000 7fda04d96740 10 client.25688 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.702+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621e7fe6410 con 0x5621e7f6fdc0 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.703+0000 7fd9daffd640 1 -- 172.21.15.102:0/3679209151 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9f4003b70 con 0x5621e7f6fdc0 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.703+0000 7fd9daffd640 1 -- 172.21.15.102:0/3679209151 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd9e40119d0 con 0x5621e7f6fdc0 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.703+0000 7fda04d96740 10 client.25688 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.703+0000 7fda04d96740 10 client.25688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.703+0000 7fda04d96740 20 client.25688 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.703+0000 7fda04d96740 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5621e7fe6940 0x5621e7fe8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.703+0000 7fda04d96740 4 client.25688 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:24:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.703+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621e7e4f8e0 con 0x5621e7fe6940 2024-04-24T23:24:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.705+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5621e7fe6940 0x5621e7fe8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.706+0000 7fd9fb7fe640 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5621e7fe6940 0x5621e7fe8d30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fdf388c9077f91c server_cookie=c72575003bb62ec3 in_seq=0 out_seq=0 2024-04-24T23:24:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.706+0000 7fd9daffd640 10 client.25688 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.786+0000 7fd9daffd640 1 -- 172.21.15.102:0/3679209151 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621e7e4f8e0 con 0x5621e7fe6940 2024-04-24T23:24:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.786+0000 7fd9daffd640 10 client.25688 handle_command_reply: tid=0 2024-04-24T23:24:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.786+0000 7fda04d96740 1 client.25688 shutdown 2024-04-24T23:24:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.786+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 msgr2=0x5621e7f6d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.786+0000 7fda04d96740 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f6d430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4010980 tx=0x7fd9e40047d0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.787+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 shutdown_connections 2024-04-24T23:24:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.787+0000 7fda04d96740 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5621e7f6fdc0 0x5621e7f6d430 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.787+0000 7fda04d96740 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5621e7fe6940 0x5621e7fe8d30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.787+0000 7fda04d96740 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621e7f65b60 0x5621e7f6cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.787+0000 7fda04d96740 1 --2- 172.21.15.102:0/3679209151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621e7f65240 0x5621e7f78900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.787+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 >> 172.21.15.102:0/3679209151 conn(0x5621e7f61560 msgr2=0x5621e7f5fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.787+0000 7fd9daffd640 0 client.25688 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.788+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 shutdown_connections 2024-04-24T23:24:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.788+0000 7fda04d96740 1 -- 172.21.15.102:0/3679209151 wait complete. 2024-04-24T23:24:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.788+0000 7fda04d96740 20 client.25688 trim_cache size 0 max 16384 2024-04-24T23:24:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:24:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:24:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:24:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:24:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.789+0000 7fda01fa9640 1 -- 172.21.15.102:0/3789312123 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd9e80a3090 msgr2=0x7fd9e80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.789+0000 7fda01fa9640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd9e80a3090 0x7fd9e80a5550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd9f402ba20 tx=0x7fd9f4000f40 comp rx=0 tx=0).stop 2024-04-24T23:24:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.790+0000 7fda01fa9640 1 -- 172.21.15.102:0/3789312123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc07de20 msgr2=0x7fd9fc1ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.790+0000 7fda01fa9640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc07de20 0x7fd9fc1ccd60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009fd0 tx=0x7fd9e400b8e0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.790+0000 7fda01fa9640 1 -- 172.21.15.102:0/3789312123 shutdown_connections 2024-04-24T23:24:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.790+0000 7fda01fa9640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc07de20 0x7fd9fc1ccd60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.790+0000 7fda01fa9640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9fc07d480 0x7fd9fc07d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.790+0000 7fda01fa9640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd9e80a3090 0x7fd9e80a5550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.790+0000 7fda01fa9640 1 --2- 172.21.15.102:0/3789312123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9fc10d0a0 0x7fd9fc083410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.790+0000 7fda01fa9640 1 -- 172.21.15.102:0/3789312123 >> 172.21.15.102:0/3789312123 conn(0x7fd9fc06c810 msgr2=0x7fd9fc10a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.791+0000 7fda01fa9640 1 -- 172.21.15.102:0/3789312123 shutdown_connections 2024-04-24T23:24:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:30.794+0000 7fda01fa9640 1 -- 172.21.15.102:0/3789312123 wait complete. 2024-04-24T23:24:31.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:31 smithi183 ceph-mon[33137]: pgmap v357: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 27 MiB/s wr, 6.77k op/s 2024-04-24T23:24:31.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:31 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:31.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:31 smithi102 ceph-mon[29216]: pgmap v357: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 27 MiB/s wr, 6.77k op/s 2024-04-24T23:24:31.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:31 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:31.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:31 smithi161 ceph-mon[32709]: pgmap v357: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 27 MiB/s wr, 6.77k op/s 2024-04-24T23:24:31.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:31 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:32 smithi183 ceph-mon[33137]: pgmap v358: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 4.74k op/s 2024-04-24T23:24:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:32 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:24:33.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:32 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:24:33.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:32 smithi102 ceph-mon[29216]: pgmap v358: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 4.74k op/s 2024-04-24T23:24:33.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:32 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:24:33.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:32 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:24:33.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:32 smithi161 ceph-mon[32709]: pgmap v358: 97 pgs: 97 active+clean; 16 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 4.74k op/s 2024-04-24T23:24:33.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:32 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:24:33.109 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:32 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:24:34.009 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:33 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:34.009 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:33 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:34.010 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:33 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:24:34.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:33 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:34.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:33 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:34.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:33 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:24:34.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:33 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:34.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:33 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:34.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:33 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:24:35.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:34 smithi183 ceph-mon[33137]: pgmap v359: 97 pgs: 97 active+clean; 16 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 26 MiB/s wr, 6.62k op/s 2024-04-24T23:24:35.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:34 smithi102 ceph-mon[29216]: pgmap v359: 97 pgs: 97 active+clean; 16 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 26 MiB/s wr, 6.62k op/s 2024-04-24T23:24:35.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:34 smithi161 ceph-mon[32709]: pgmap v359: 97 pgs: 97 active+clean; 16 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 26 MiB/s wr, 6.62k op/s 2024-04-24T23:24:37.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:36 smithi183 ceph-mon[33137]: pgmap v360: 97 pgs: 97 active+clean; 16 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 5.27k op/s 2024-04-24T23:24:37.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:36 smithi102 ceph-mon[29216]: pgmap v360: 97 pgs: 97 active+clean; 16 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 5.27k op/s 2024-04-24T23:24:37.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:36 smithi161 ceph-mon[32709]: pgmap v360: 97 pgs: 97 active+clean; 16 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 5.27k op/s 2024-04-24T23:24:39.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:38 smithi183 ceph-mon[33137]: pgmap v361: 97 pgs: 97 active+clean; 16 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2024-04-24T23:24:39.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:38 smithi102 ceph-mon[29216]: pgmap v361: 97 pgs: 97 active+clean; 16 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2024-04-24T23:24:39.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:38 smithi161 ceph-mon[32709]: pgmap v361: 97 pgs: 97 active+clean; 16 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2024-04-24T23:24:41.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:40 smithi183 ceph-mon[33137]: pgmap v362: 97 pgs: 97 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 26 MiB/s wr, 6.72k op/s 2024-04-24T23:24:41.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:40 smithi102 ceph-mon[29216]: pgmap v362: 97 pgs: 97 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 26 MiB/s wr, 6.72k op/s 2024-04-24T23:24:41.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:40 smithi161 ceph-mon[32709]: pgmap v362: 97 pgs: 97 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 26 MiB/s wr, 6.72k op/s 2024-04-24T23:24:43.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:42 smithi183 ceph-mon[33137]: pgmap v363: 97 pgs: 97 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 19 MiB/s wr, 4.76k op/s 2024-04-24T23:24:43.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:42 smithi102 ceph-mon[29216]: pgmap v363: 97 pgs: 97 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 19 MiB/s wr, 4.76k op/s 2024-04-24T23:24:43.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:42 smithi161 ceph-mon[32709]: pgmap v363: 97 pgs: 97 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 19 MiB/s wr, 4.76k op/s 2024-04-24T23:24:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.009+0000 7fd58d226640 1 -- 172.21.15.102:0/4193816024 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580030c60 con 0x7fd5881050d0 2024-04-24T23:24:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.010+0000 7fd59048c640 1 -- 172.21.15.102:0/4193816024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5881050d0 msgr2=0x7fd58810cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.010+0000 7fd59048c640 1 --2- 172.21.15.102:0/4193816024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5881050d0 0x7fd58810cca0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd580009ca0 tx=0x7fd58002a8b0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.011+0000 7fd59048c640 1 -- 172.21.15.102:0/4193816024 shutdown_connections 2024-04-24T23:24:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.011+0000 7fd59048c640 1 --2- 172.21.15.102:0/4193816024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5881050d0 0x7fd58810cca0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.011+0000 7fd59048c640 1 --2- 172.21.15.102:0/4193816024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5881009b0 0x7fd5881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.011+0000 7fd59048c640 1 --2- 172.21.15.102:0/4193816024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd588100000 0x7fd5881003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.011+0000 7fd59048c640 1 -- 172.21.15.102:0/4193816024 >> 172.21.15.102:0/4193816024 conn(0x7fd588076bd0 msgr2=0x7fd5880fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.012+0000 7fd59048c640 1 -- 172.21.15.102:0/4193816024 shutdown_connections 2024-04-24T23:24:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.012+0000 7fd59048c640 1 -- 172.21.15.102:0/4193816024 wait complete. 2024-04-24T23:24:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd59048c640 1 Processor -- start 2024-04-24T23:24:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd59048c640 1 -- start start 2024-04-24T23:24:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd59048c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd588100000 0x7fd588072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd59048c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5881050d0 0x7fd5880733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd59048c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd58806c0a0 0x7fd58806c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd59048c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd588111a10 con 0x7fd5881050d0 2024-04-24T23:24:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd59048c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd588111890 con 0x7fd588100000 2024-04-24T23:24:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd59048c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd588111b90 con 0x7fd58806c0a0 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd58e228640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd588100000 0x7fd588072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd58e228640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd588100000 0x7fd588072e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:57566/0 (socket says 172.21.15.102:57566) 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.013+0000 7fd58e228640 1 -- 172.21.15.102:0/2538558222 learned_addr learned my addr 172.21.15.102:0/2538558222 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.014+0000 7fd58ea29640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd58806c0a0 0x7fd58806c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.014+0000 7fd58e228640 1 -- 172.21.15.102:0/2538558222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd58806c0a0 msgr2=0x7fd58806c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.014+0000 7fd58e228640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd58806c0a0 0x7fd58806c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.014+0000 7fd58e228640 1 -- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5881050d0 msgr2=0x7fd5880733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.014+0000 7fd58da27640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5881050d0 0x7fd5880733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.014+0000 7fd58e228640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5881050d0 0x7fd5880733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.014+0000 7fd58e228640 1 -- 172.21.15.102:0/2538558222 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd588070990 con 0x7fd588100000 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.014+0000 7fd58da27640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5881050d0 0x7fd5880733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.014+0000 7fd58e228640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd588100000 0x7fd588072e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd58400e9e0 tx=0x7fd58400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.016+0000 7fd57f7fe640 1 -- 172.21.15.102:0/2538558222 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58400cd50 con 0x7fd588100000 2024-04-24T23:24:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.016+0000 7fd59048c640 1 -- 172.21.15.102:0/2538558222 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd588070c80 con 0x7fd588100000 2024-04-24T23:24:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.016+0000 7fd59048c640 1 -- 172.21.15.102:0/2538558222 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd588071090 con 0x7fd588100000 2024-04-24T23:24:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.016+0000 7fd57f7fe640 1 -- 172.21.15.102:0/2538558222 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5840035f0 con 0x7fd588100000 2024-04-24T23:24:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.016+0000 7fd57f7fe640 1 -- 172.21.15.102:0/2538558222 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584010690 con 0x7fd588100000 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.022+0000 7fd593198740 1 Processor -- start 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.022+0000 7fd593198740 1 -- start start 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.022+0000 7fd593198740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed17069620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.022+0000 7fd593198740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.022+0000 7fd593198740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17076620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.022+0000 7fd593198740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed16fcdcb0 con 0x55ed17069b60 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.022+0000 7fd593198740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed16f765b0 con 0x55ed17074230 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.022+0000 7fd593198740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed16fc9c70 con 0x55ed17069240 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58ea29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed17069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58da27640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17076620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58ea29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed17069620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36756/0 (socket says 172.21.15.102:36756) 2024-04-24T23:24:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58ea29640 1 -- 172.21.15.102:0/2619671092 learned_addr learned my addr 172.21.15.102:0/2619671092 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58da27640 1 -- 172.21.15.102:0/2619671092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 msgr2=0x55ed17069620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58da27640 1 --2- 172.21.15.102:0/2619671092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed17069620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58da27640 1 -- 172.21.15.102:0/2619671092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 msgr2=0x55ed17073cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58da27640 1 --2- 172.21.15.102:0/2619671092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17073cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58da27640 1 -- 172.21.15.102:0/2619671092 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed16ce2210 con 0x55ed17074230 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd57f7fe640 1 -- 172.21.15.102:0/2538558222 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd5840037a0 con 0x7fd588100000 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd58da27640 1 --2- 172.21.15.102:0/2619671092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17076620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd58806d740 tx=0x7fd578010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaadc9c2c47e9c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd57cbfe640 1 -- 172.21.15.102:0/2619671092 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578009e50 con 0x55ed17074230 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd57cbfe640 1 -- 172.21.15.102:0/2619671092 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5780035f0 con 0x55ed17074230 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.023+0000 7fd57cbfe640 1 -- 172.21.15.102:0/2619671092 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5780044e0 con 0x55ed17074230 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.024+0000 7fd57f7fe640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd5740ab7e0 0x7fd5740adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.024+0000 7fd57f7fe640 1 -- 172.21.15.102:0/2538558222 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7fd5840cf730 con 0x7fd588100000 2024-04-24T23:24:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.024+0000 7fd58ea29640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd5740ab7e0 0x7fd5740adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.024+0000 7fd593198740 1 -- 172.21.15.102:0/2619671092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 msgr2=0x55ed17076620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.024+0000 7fd593198740 1 --2- 172.21.15.102:0/2619671092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17076620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd58806d740 tx=0x7fd578010740 comp rx=0 tx=0).stop 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.025+0000 7fd593198740 1 -- 172.21.15.102:0/2619671092 shutdown_connections 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.025+0000 7fd593198740 1 --2- 172.21.15.102:0/2619671092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17076620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.025+0000 7fd593198740 1 --2- 172.21.15.102:0/2619671092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.025+0000 7fd593198740 1 --2- 172.21.15.102:0/2619671092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed17069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.025+0000 7fd593198740 1 -- 172.21.15.102:0/2619671092 >> 172.21.15.102:0/2619671092 conn(0x55ed17065560 msgr2=0x55ed17065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.025+0000 7fd58ea29640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd5740ab7e0 0x7fd5740adca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd580009a60 tx=0x7fd580004a40 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.026+0000 7fd593198740 1 -- 172.21.15.102:0/2619671092 shutdown_connections 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.026+0000 7fd593198740 1 -- 172.21.15.102:0/2619671092 wait complete. 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.026+0000 7fd593198740 1 -- start start 2024-04-24T23:24:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.026+0000 7fd593198740 1 Processor -- start 2024-04-24T23:24:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.027+0000 7fd593198740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.027+0000 7fd593198740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17069b60 0x55ed17071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.027+0000 7fd593198740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17074230 0x55ed17071c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.027+0000 7fd593198740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed17077020 con 0x55ed17074230 2024-04-24T23:24:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.027+0000 7fd593198740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed16f89b50 con 0x55ed17069b60 2024-04-24T23:24:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.027+0000 7fd593198740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed170771a0 con 0x55ed17069240 2024-04-24T23:24:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.027+0000 7fd593198740 10 client.0 _ll_register_callbacks cb 0x55ed16fc4ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.028+0000 7fd58e228640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.028+0000 7fd58e228640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36770/0 (socket says 172.21.15.102:36770) 2024-04-24T23:24:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.028+0000 7fd58e228640 1 -- 172.21.15.102:0/2633754004 learned_addr learned my addr 172.21.15.102:0/2633754004 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.028+0000 7fd58da27640 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17069b60 0x55ed17071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.028+0000 7fd58ea29640 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17074230 0x55ed17071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.029+0000 7fd58ea29640 1 -- 172.21.15.102:0/2633754004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 msgr2=0x55ed170711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.029+0000 7fd58ea29640 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.029+0000 7fd58ea29640 1 -- 172.21.15.102:0/2633754004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17069b60 msgr2=0x55ed17071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.029+0000 7fd58ea29640 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17069b60 0x55ed17071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.029+0000 7fd58ea29640 1 -- 172.21.15.102:0/2633754004 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed16dae620 con 0x55ed17074230 2024-04-24T23:24:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.030+0000 7fd58ea29640 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17074230 0x55ed17071c40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd580031040 tx=0x7fd58002a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f203a575d2e0226 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.030+0000 7fd571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:24:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.031+0000 7fd571ffb640 1 -- 172.21.15.102:0/2633754004 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580053440 con 0x55ed17074230 2024-04-24T23:24:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.031+0000 7fd571ffb640 1 -- 172.21.15.102:0/2633754004 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7fd5800535e0 con 0x55ed17074230 2024-04-24T23:24:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.031+0000 7fd593198740 1 -- 172.21.15.102:0/2633754004 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5780035f0 con 0x55ed17074230 2024-04-24T23:24:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.032+0000 7fd571ffb640 1 -- 172.21.15.102:0/2633754004 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd580053820 con 0x55ed17074230 2024-04-24T23:24:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.035+0000 7fd593198740 10 client.16290 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.035+0000 7fd593198740 1 -- 172.21.15.102:0/2633754004 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed170832b0 con 0x55ed17074230 2024-04-24T23:24:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.036+0000 7fd571ffb640 1 -- 172.21.15.102:0/2633754004 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5780035f0 con 0x55ed17074230 2024-04-24T23:24:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.036+0000 7fd571ffb640 1 -- 172.21.15.102:0/2633754004 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd580051020 con 0x55ed17074230 2024-04-24T23:24:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.036+0000 7fd593198740 10 client.16290 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.036+0000 7fd593198740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.036+0000 7fd593198740 20 client.16290 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.036+0000 7fd593198740 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed170847c0 0x55ed17086bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.036+0000 7fd593198740 4 client.16290 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.036+0000 7fd593198740 1 -- 172.21.15.102:0/2633754004 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed16fdb1f0 con 0x55ed170847c0 2024-04-24T23:24:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.037+0000 7fd58e228640 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed170847c0 0x55ed17086bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.038+0000 7fd571ffb640 10 client.16290 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.038+0000 7fd58e228640 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed170847c0 0x55ed17086bd0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4997aa57e238ee7 server_cookie=d66c69747550859c in_seq=0 out_seq=0 2024-04-24T23:24:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.039+0000 7fd571ffb640 1 -- 172.21.15.102:0/2633754004 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed16fdb1f0 con 0x55ed170847c0 2024-04-24T23:24:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.039+0000 7fd571ffb640 10 client.16290 handle_command_reply: tid=0 2024-04-24T23:24:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.039+0000 7fd593198740 1 client.16290 shutdown 2024-04-24T23:24:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 -- 172.21.15.102:0/2633754004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17074230 msgr2=0x55ed17071c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17074230 0x55ed17071c40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd580031040 tx=0x7fd58002a8b0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 -- 172.21.15.102:0/2633754004 shutdown_connections 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed170847c0 0x55ed17086bd0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17074230 0x55ed17071c40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17069b60 0x55ed17071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 --2- 172.21.15.102:0/2633754004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 -- 172.21.15.102:0/2633754004 >> 172.21.15.102:0/2633754004 conn(0x55ed17065560 msgr2=0x55ed1706b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd571ffb640 0 client.16290 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 -- 172.21.15.102:0/2633754004 shutdown_connections 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 1 -- 172.21.15.102:0/2633754004 wait complete. 2024-04-24T23:24:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.040+0000 7fd593198740 20 client.16290 trim_cache size 0 max 16384 2024-04-24T23:24:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.072+0000 7fd593198740 1 Processor -- start 2024-04-24T23:24:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd593198740 1 -- start start 2024-04-24T23:24:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd593198740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170738f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd593198740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17064d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd593198740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd593198740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c011220 con 0x55ed17069b60 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd593198740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c00eaf0 con 0x55ed17074230 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd593198740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c00c960 con 0x55ed17069240 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd58e228640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd58e228640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170738f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36782/0 (socket says 172.21.15.102:36782) 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd58e228640 1 -- 172.21.15.102:0/2617408078 learned_addr learned my addr 172.21.15.102:0/2617408078 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.074+0000 7fd58ea29640 1 --2- 172.21.15.102:0/2617408078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.075+0000 7fd58e228640 1 -- 172.21.15.102:0/2617408078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 msgr2=0x55ed17071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.075+0000 7fd58e228640 1 --2- 172.21.15.102:0/2617408078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.075+0000 7fd58e228640 1 -- 172.21.15.102:0/2617408078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 msgr2=0x55ed17064d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.075+0000 7fd58e228640 1 --2- 172.21.15.102:0/2617408078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17064d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.075+0000 7fd58e228640 1 -- 172.21.15.102:0/2617408078 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed170725b0 con 0x55ed17069240 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.075+0000 7fd58ea29640 1 --2- 172.21.15.102:0/2617408078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.075+0000 7fd58e228640 1 --2- 172.21.15.102:0/2617408078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170738f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd584098580 tx=0x7fd584003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e76fbd3d9426ebcf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.075+0000 7fd573fff640 1 -- 172.21.15.102:0/2617408078 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5840ba070 con 0x55ed17069240 2024-04-24T23:24:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.075+0000 7fd573fff640 1 -- 172.21.15.102:0/2617408078 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd58409ff60 con 0x55ed17069240 2024-04-24T23:24:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd573fff640 1 -- 172.21.15.102:0/2617408078 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584097900 con 0x55ed17069240 2024-04-24T23:24:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 -- 172.21.15.102:0/2617408078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 msgr2=0x55ed170738f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 --2- 172.21.15.102:0/2617408078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170738f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd584098580 tx=0x7fd584003e10 comp rx=0 tx=0).stop 2024-04-24T23:24:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 -- 172.21.15.102:0/2617408078 shutdown_connections 2024-04-24T23:24:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 --2- 172.21.15.102:0/2617408078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 --2- 172.21.15.102:0/2617408078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17064d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 --2- 172.21.15.102:0/2617408078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed170738f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 -- 172.21.15.102:0/2617408078 >> 172.21.15.102:0/2617408078 conn(0x55ed17065560 msgr2=0x55ed1706b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 -- 172.21.15.102:0/2617408078 shutdown_connections 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 -- 172.21.15.102:0/2617408078 wait complete. 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.076+0000 7fd593198740 1 -- start start 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd593198740 1 Processor -- start 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd593198740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed1707c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd593198740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17077a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd593198740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17077f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd593198740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c00cfb0 con 0x55ed17069b60 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd593198740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c00e420 con 0x55ed17074230 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd593198740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c00d680 con 0x55ed17069240 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd593198740 10 client.0 _ll_register_callbacks cb 0x7fd580009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd58da27640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17077a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd58da27640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17077a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54622/0 (socket says 172.21.15.102:54622) 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd58da27640 1 -- 172.21.15.102:0/1105091125 learned_addr learned my addr 172.21.15.102:0/1105091125 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd58ea29640 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17077f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.077+0000 7fd58e228640 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed1707c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd58ea29640 1 -- 172.21.15.102:0/1105091125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 msgr2=0x55ed1707c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd58ea29640 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed1707c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd58ea29640 1 -- 172.21.15.102:0/1105091125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 msgr2=0x55ed17077a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd58ea29640 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17077a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd58ea29640 1 -- 172.21.15.102:0/1105091125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed170785f0 con 0x55ed17074230 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd58da27640 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17077a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd58ea29640 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17077f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd580004ab0 tx=0x7fd5800536f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b457cbfc0ec5b105 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd571ffb640 1 -- 172.21.15.102:0/1105091125 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580053dd0 con 0x55ed17074230 2024-04-24T23:24:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd571ffb640 1 -- 172.21.15.102:0/1105091125 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7fd580049420 con 0x55ed17074230 2024-04-24T23:24:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd593198740 1 -- 172.21.15.102:0/1105091125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58409ff60 con 0x55ed17074230 2024-04-24T23:24:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd571ffb640 1 -- 172.21.15.102:0/1105091125 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd58006b050 con 0x55ed17074230 2024-04-24T23:24:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd593198740 10 client.25706 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd593198740 1 -- 172.21.15.102:0/1105091125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed170eac10 con 0x55ed17074230 2024-04-24T23:24:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.078+0000 7fd571ffb640 1 -- 172.21.15.102:0/1105091125 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd58409ff60 con 0x55ed17074230 2024-04-24T23:24:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.080+0000 7fd571ffb640 1 -- 172.21.15.102:0/1105091125 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd580051040 con 0x55ed17074230 2024-04-24T23:24:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.081+0000 7fd593198740 10 client.25706 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.081+0000 7fd593198740 10 client.25706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.081+0000 7fd593198740 20 client.25706 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.081+0000 7fd593198740 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed170eb390 0x55ed170ed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.082+0000 7fd593198740 4 client.25706 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:24:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.082+0000 7fd593198740 1 -- 172.21.15.102:0/1105091125 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed16fd5540 con 0x55ed170eb390 2024-04-24T23:24:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.082+0000 7fd58e228640 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed170eb390 0x55ed170ed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.083+0000 7fd571ffb640 10 client.25706 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.083+0000 7fd58e228640 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed170eb390 0x55ed170ed7b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a91569a4d55de21 server_cookie=30a289a4bb6bac3c in_seq=0 out_seq=0 2024-04-24T23:24:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.191+0000 7fd571ffb640 1 -- 172.21.15.102:0/1105091125 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed16fd5540 con 0x55ed170eb390 2024-04-24T23:24:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.191+0000 7fd571ffb640 10 client.25706 handle_command_reply: tid=0 2024-04-24T23:24:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.191+0000 7fd593198740 1 client.25706 shutdown 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 -- 172.21.15.102:0/1105091125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 msgr2=0x55ed17077f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17077f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd580004ab0 tx=0x7fd5800536f0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 -- 172.21.15.102:0/1105091125 shutdown_connections 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed17074230 0x55ed17077f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55ed170eb390 0x55ed170ed7b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed17069b60 0x55ed17077a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 --2- 172.21.15.102:0/1105091125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed17069240 0x55ed1707c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 -- 172.21.15.102:0/1105091125 >> 172.21.15.102:0/1105091125 conn(0x55ed17065560 msgr2=0x55ed17072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd571ffb640 0 client.25706 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 -- 172.21.15.102:0/1105091125 shutdown_connections 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.192+0000 7fd593198740 1 -- 172.21.15.102:0/1105091125 wait complete. 2024-04-24T23:24:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd593198740 20 client.25706 trim_cache size 0 max 16384 2024-04-24T23:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:24:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:24:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd59048c640 1 -- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd5740ab7e0 msgr2=0x7fd5740adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd59048c640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd5740ab7e0 0x7fd5740adca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd580009a60 tx=0x7fd580004a40 comp rx=0 tx=0).stop 2024-04-24T23:24:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd59048c640 1 -- 172.21.15.102:0/2538558222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd588100000 msgr2=0x7fd588072e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd59048c640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd588100000 0x7fd588072e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd58400e9e0 tx=0x7fd58400eea0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd59048c640 1 -- 172.21.15.102:0/2538558222 shutdown_connections 2024-04-24T23:24:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd59048c640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd58806c0a0 0x7fd58806c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd59048c640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd5740ab7e0 0x7fd5740adca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd59048c640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5881050d0 0x7fd5880733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.193+0000 7fd59048c640 1 --2- 172.21.15.102:0/2538558222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd588100000 0x7fd588072e70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.194+0000 7fd59048c640 1 -- 172.21.15.102:0/2538558222 >> 172.21.15.102:0/2538558222 conn(0x7fd588076bd0 msgr2=0x7fd5880fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.194+0000 7fd59048c640 1 -- 172.21.15.102:0/2538558222 shutdown_connections 2024-04-24T23:24:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:44.194+0000 7fd59048c640 1 -- 172.21.15.102:0/2538558222 wait complete. 2024-04-24T23:24:45.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:44 smithi183 ceph-mon[33137]: pgmap v364: 97 pgs: 97 active+clean; 17 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 26 MiB/s wr, 6.62k op/s 2024-04-24T23:24:45.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:44 smithi102 ceph-mon[29216]: pgmap v364: 97 pgs: 97 active+clean; 17 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 26 MiB/s wr, 6.62k op/s 2024-04-24T23:24:45.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:44 smithi161 ceph-mon[32709]: pgmap v364: 97 pgs: 97 active+clean; 17 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 26 MiB/s wr, 6.62k op/s 2024-04-24T23:24:47.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:46 smithi183 ceph-mon[33137]: pgmap v365: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 21 MiB/s wr, 5.28k op/s 2024-04-24T23:24:47.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:46 smithi102 ceph-mon[29216]: pgmap v365: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 21 MiB/s wr, 5.28k op/s 2024-04-24T23:24:47.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:46 smithi161 ceph-mon[32709]: pgmap v365: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 21 MiB/s wr, 5.28k op/s 2024-04-24T23:24:49.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:48 smithi183 ceph-mon[33137]: pgmap v366: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 21 MiB/s wr, 5.30k op/s 2024-04-24T23:24:49.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:48 smithi102 ceph-mon[29216]: pgmap v366: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 21 MiB/s wr, 5.30k op/s 2024-04-24T23:24:49.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:48 smithi161 ceph-mon[32709]: pgmap v366: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 21 MiB/s wr, 5.30k op/s 2024-04-24T23:24:51.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:50 smithi183 ceph-mon[33137]: pgmap v367: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 27 MiB/s wr, 6.81k op/s 2024-04-24T23:24:51.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:50 smithi102 ceph-mon[29216]: pgmap v367: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 27 MiB/s wr, 6.81k op/s 2024-04-24T23:24:51.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:50 smithi161 ceph-mon[32709]: pgmap v367: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 27 MiB/s wr, 6.81k op/s 2024-04-24T23:24:53.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:52 smithi183 ceph-mon[33137]: pgmap v368: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2024-04-24T23:24:53.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:52 smithi102 ceph-mon[29216]: pgmap v368: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2024-04-24T23:24:53.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:52 smithi161 ceph-mon[32709]: pgmap v368: 97 pgs: 97 active+clean; 17 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2024-04-24T23:24:55.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:54 smithi183 ceph-mon[33137]: pgmap v369: 97 pgs: 97 active+clean; 17 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 27 MiB/s wr, 6.86k op/s 2024-04-24T23:24:55.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:54 smithi102 ceph-mon[29216]: pgmap v369: 97 pgs: 97 active+clean; 17 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 27 MiB/s wr, 6.86k op/s 2024-04-24T23:24:55.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:54 smithi161 ceph-mon[32709]: pgmap v369: 97 pgs: 97 active+clean; 17 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 27 MiB/s wr, 6.86k op/s 2024-04-24T23:24:57.324 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:57 smithi102 ceph-mon[29216]: pgmap v370: 97 pgs: 97 active+clean; 17 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:24:57.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:57 smithi161 ceph-mon[32709]: pgmap v370: 97 pgs: 97 active+clean; 17 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:24:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.409+0000 7f92da95a640 1 -- 172.21.15.102:0/3476232327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 msgr2=0x7f92d4108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.409+0000 7f92da95a640 1 --2- 172.21.15.102:0/3476232327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 0x7f92d4108740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009ca0 tx=0x7f92cc02a950 comp rx=0 tx=0).stop 2024-04-24T23:24:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.409+0000 7f92da95a640 1 -- 172.21.15.102:0/3476232327 shutdown_connections 2024-04-24T23:24:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.409+0000 7f92da95a640 1 --2- 172.21.15.102:0/3476232327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92d411af20 0x7f92d411d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.409+0000 7f92da95a640 1 --2- 172.21.15.102:0/3476232327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92d4110310 0x7f92d4108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.409+0000 7f92da95a640 1 --2- 172.21.15.102:0/3476232327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 0x7f92d4108740 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.409+0000 7f92da95a640 1 -- 172.21.15.102:0/3476232327 >> 172.21.15.102:0/3476232327 conn(0x7f92d406c810 msgr2=0x7f92d406cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.413+0000 7f92da95a640 1 -- 172.21.15.102:0/3476232327 shutdown_connections 2024-04-24T23:24:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.414+0000 7f92da95a640 1 -- 172.21.15.102:0/3476232327 wait complete. 2024-04-24T23:24:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.414+0000 7f92da95a640 1 Processor -- start 2024-04-24T23:24:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92da95a640 1 -- start start 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92da95a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 0x7f92d41a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92da95a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92d411af20 0x7f92d41a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92da95a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92d41ab660 0x7f92d41abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92da95a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41208e0 con 0x7f92d411af20 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92da95a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4120760 con 0x7f92d41ab660 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92da95a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4120a60 con 0x7f92d410ff30 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 0x7f92d41a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 0x7f92d41a6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:54458/0 (socket says 172.21.15.102:54458) 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92d8ef7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92d41ab660 0x7f92d41abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.415+0000 7f92d3fff640 1 -- 172.21.15.102:0/2578717994 learned_addr learned my addr 172.21.15.102:0/2578717994 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.416+0000 7f92d3fff640 1 -- 172.21.15.102:0/2578717994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92d41ab660 msgr2=0x7f92d41abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.416+0000 7f92d3fff640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92d41ab660 0x7f92d41abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.416+0000 7f92d3fff640 1 -- 172.21.15.102:0/2578717994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92d411af20 msgr2=0x7f92d41a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.416+0000 7f92d3fff640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92d411af20 0x7f92d41a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.416+0000 7f92d3fff640 1 -- 172.21.15.102:0/2578717994 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d41ac150 con 0x7f92d410ff30 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.417+0000 7f92d3fff640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 0x7f92d41a6da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005e30 tx=0x7f92cc003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.417+0000 7f92d17fa640 1 -- 172.21.15.102:0/2578717994 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc034040 con 0x7f92d410ff30 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.417+0000 7f92da95a640 1 -- 172.21.15.102:0/2578717994 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d41ac3e0 con 0x7f92d410ff30 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.417+0000 7f92da95a640 1 -- 172.21.15.102:0/2578717994 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d41b3c00 con 0x7f92d410ff30 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.417+0000 7f92d17fa640 1 -- 172.21.15.102:0/2578717994 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92cc030850 con 0x7f92d410ff30 2024-04-24T23:24:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.417+0000 7f92d17fa640 1 -- 172.21.15.102:0/2578717994 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004750 con 0x7f92d410ff30 2024-04-24T23:24:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.421+0000 7f92d17fa640 1 -- 172.21.15.102:0/2578717994 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f92cc037020 con 0x7f92d410ff30 2024-04-24T23:24:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.422+0000 7f92d17fa640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92bc0a2ea0 0x7f92bc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.423+0000 7f92d17fa640 1 -- 172.21.15.102:0/2578717994 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7f92cc0f1230 con 0x7f92d410ff30 2024-04-24T23:24:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.423+0000 7f92dd661740 1 Processor -- start 2024-04-24T23:24:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.423+0000 7f92d37fe640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92bc0a2ea0 0x7f92bc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.423+0000 7f92dd661740 1 -- start start 2024-04-24T23:24:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92dd661740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fb240 0x564f154fb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92dd661740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15505cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92dd661740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 0x564f15508620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92dd661740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564f1545fcb0 con 0x564f154fb240 2024-04-24T23:24:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92dd661740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564f154085b0 con 0x564f154fbb60 2024-04-24T23:24:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92dd661740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564f1545bc70 con 0x564f15506230 2024-04-24T23:24:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d8ef7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 0x564f15508620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d8ef7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 0x564f15508620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:54468/0 (socket says 172.21.15.102:54468) 2024-04-24T23:24:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d8ef7640 1 -- 172.21.15.102:0/3769290243 learned_addr learned my addr 172.21.15.102:0/3769290243 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d37fe640 1 --2- 172.21.15.102:0/3769290243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15505cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d8ef7640 1 -- 172.21.15.102:0/3769290243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 msgr2=0x564f15505cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d8ef7640 1 --2- 172.21.15.102:0/3769290243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15505cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d8ef7640 1 -- 172.21.15.102:0/3769290243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fb240 msgr2=0x564f154fb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d8ef7640 1 --2- 172.21.15.102:0/3769290243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fb240 0x564f154fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d8ef7640 1 -- 172.21.15.102:0/3769290243 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f15174210 con 0x564f15506230 2024-04-24T23:24:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.424+0000 7f92d37fe640 1 --2- 172.21.15.102:0/3769290243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15505cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.425+0000 7f92d8ef7640 1 --2- 172.21.15.102:0/3769290243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 0x564f15508620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92c400b780 tx=0x7f92c400bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9221050523461cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.425+0000 7f92d37fe640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92bc0a2ea0 0x7f92bc0a5360 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f92c80058c0 tx=0x7f92c8005850 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.425+0000 7f92ba7fc640 1 -- 172.21.15.102:0/3769290243 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c400d950 con 0x564f15506230 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.425+0000 7f92ba7fc640 1 -- 172.21.15.102:0/3769290243 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92c4014430 con 0x564f15506230 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.425+0000 7f92ba7fc640 1 -- 172.21.15.102:0/3769290243 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40123e0 con 0x564f15506230 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.426+0000 7f92dd661740 1 -- 172.21.15.102:0/3769290243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 msgr2=0x564f15508620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.426+0000 7f92dd661740 1 --2- 172.21.15.102:0/3769290243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 0x564f15508620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92c400b780 tx=0x7f92c400bc40 comp rx=0 tx=0).stop 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.426+0000 7f92dd661740 1 -- 172.21.15.102:0/3769290243 shutdown_connections 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.426+0000 7f92dd661740 1 --2- 172.21.15.102:0/3769290243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 0x564f15508620 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.426+0000 7f92dd661740 1 --2- 172.21.15.102:0/3769290243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15505cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.426+0000 7f92dd661740 1 --2- 172.21.15.102:0/3769290243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fb240 0x564f154fb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.426+0000 7f92dd661740 1 -- 172.21.15.102:0/3769290243 >> 172.21.15.102:0/3769290243 conn(0x564f154f7560 msgr2=0x564f154f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.428+0000 7f92dd661740 1 -- 172.21.15.102:0/3769290243 shutdown_connections 2024-04-24T23:24:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.428+0000 7f92dd661740 1 -- 172.21.15.102:0/3769290243 wait complete. 2024-04-24T23:24:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.428+0000 7f92dd661740 1 -- start start 2024-04-24T23:24:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92dd661740 1 Processor -- start 2024-04-24T23:24:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92dd661740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fb240 0x564f15503180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92dd661740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fbb60 0x564f155036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92dd661740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 0x564f15503c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92dd661740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564f15509020 con 0x564f154fbb60 2024-04-24T23:24:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92dd661740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564f1541bb50 con 0x564f154fb240 2024-04-24T23:24:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92dd661740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564f155091a0 con 0x564f15506230 2024-04-24T23:24:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92dd661740 10 client.0 _ll_register_callbacks cb 0x564f15456ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fb240 0x564f15503180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fb240 0x564f15503180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:58038/0 (socket says 172.21.15.102:58038) 2024-04-24T23:24:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.429+0000 7f92d3fff640 1 -- 172.21.15.102:0/4241233299 learned_addr learned my addr 172.21.15.102:0/4241233299 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.430+0000 7f92d3fff640 1 -- 172.21.15.102:0/4241233299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 msgr2=0x564f15503c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:24:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.430+0000 7f92d3fff640 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 0x564f15503c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.430+0000 7f92d3fff640 1 -- 172.21.15.102:0/4241233299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fbb60 msgr2=0x564f155036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.430+0000 7f92d3fff640 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fbb60 0x564f155036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.430+0000 7f92d3fff640 1 -- 172.21.15.102:0/4241233299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f15240620 con 0x564f154fb240 2024-04-24T23:24:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.430+0000 7f92d3fff640 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fb240 0x564f15503180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0043a0 tx=0x7f92cc0bd300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f2afab140dfdca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.431+0000 7f92b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:24:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.431+0000 7f92b37fe640 1 -- 172.21.15.102:0/4241233299 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc02c040 con 0x564f154fb240 2024-04-24T23:24:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.431+0000 7f92dd661740 1 -- 172.21.15.102:0/4241233299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c4014430 con 0x564f154fb240 2024-04-24T23:24:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.431+0000 7f92b37fe640 1 -- 172.21.15.102:0/4241233299 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7f92cc0a80e0 con 0x564f154fb240 2024-04-24T23:24:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.431+0000 7f92b37fe640 1 -- 172.21.15.102:0/4241233299 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f92cc0bdec0 con 0x564f154fb240 2024-04-24T23:24:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.432+0000 7f92dd661740 10 client.25715 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.432+0000 7f92dd661740 1 -- 172.21.15.102:0/4241233299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f155261e0 con 0x564f154fb240 2024-04-24T23:24:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.432+0000 7f92b37fe640 1 -- 172.21.15.102:0/4241233299 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92c4014430 con 0x564f154fb240 2024-04-24T23:24:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.432+0000 7f92b37fe640 1 -- 172.21.15.102:0/4241233299 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f92cc0d1ce0 con 0x564f154fb240 2024-04-24T23:24:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.433+0000 7f92dd661740 10 client.25715 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.433+0000 7f92dd661740 10 client.25715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.433+0000 7f92dd661740 20 client.25715 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.433+0000 7f92dd661740 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x564f155276f0 0x564f15529b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.433+0000 7f92dd661740 4 client.25715 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:24:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.433+0000 7f92dd661740 1 -- 172.21.15.102:0/4241233299 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f1546d1f0 con 0x564f155276f0 2024-04-24T23:24:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.433+0000 7f92d37fe640 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x564f155276f0 0x564f15529b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.434+0000 7f92b37fe640 10 client.25715 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.434+0000 7f92d37fe640 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x564f155276f0 0x564f15529b50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab08fde50ab3e81 server_cookie=8501092dfa04dac in_seq=0 out_seq=0 2024-04-24T23:24:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.435+0000 7f92b37fe640 1 -- 172.21.15.102:0/4241233299 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f1546d1f0 con 0x564f155276f0 2024-04-24T23:24:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.435+0000 7f92b37fe640 10 client.25715 handle_command_reply: tid=0 2024-04-24T23:24:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.435+0000 7f92dd661740 1 client.25715 shutdown 2024-04-24T23:24:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.437+0000 7f92dd661740 1 -- 172.21.15.102:0/4241233299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fb240 msgr2=0x564f15503180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.437+0000 7f92dd661740 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fb240 0x564f15503180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0043a0 tx=0x7f92cc0bd300 comp rx=0 tx=0).stop 2024-04-24T23:24:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.442+0000 7f92dd661740 1 -- 172.21.15.102:0/4241233299 shutdown_connections 2024-04-24T23:24:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.442+0000 7f92dd661740 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f15506230 0x564f15503c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.442+0000 7f92dd661740 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x564f155276f0 0x564f15529b50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.442+0000 7f92dd661740 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fbb60 0x564f155036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.442+0000 7f92dd661740 1 --2- 172.21.15.102:0/4241233299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fb240 0x564f15503180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.442+0000 7f92dd661740 1 -- 172.21.15.102:0/4241233299 >> 172.21.15.102:0/4241233299 conn(0x564f154f7560 msgr2=0x564f154fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.442+0000 7f92b37fe640 0 client.25715 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.443+0000 7f92dd661740 1 -- 172.21.15.102:0/4241233299 shutdown_connections 2024-04-24T23:24:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.443+0000 7f92dd661740 1 -- 172.21.15.102:0/4241233299 wait complete. 2024-04-24T23:24:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.443+0000 7f92dd661740 20 client.25715 trim_cache size 0 max 16384 2024-04-24T23:24:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.480+0000 7f92dd661740 1 Processor -- start 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.480+0000 7f92dd661740 1 -- start start 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92dd661740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154f7560 0x564f15511f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92dd661740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fb240 0x564f15501a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92dd661740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15503a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92d37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fb240 0x564f15501a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154f7560 0x564f15511f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92d37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fb240 0x564f15501a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:54490/0 (socket says 172.21.15.102:54490) 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154f7560 0x564f15511f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53128/0 (socket says 172.21.15.102:53128) 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92d8ef7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15503a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92d8ef7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15503a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:58044/0 (socket says 172.21.15.102:58044) 2024-04-24T23:24:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.481+0000 7f92d37fe640 1 -- 172.21.15.102:0/704234698 learned_addr learned my addr 172.21.15.102:0/704234698 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.482+0000 7f92dd661740 1 -- 172.21.15.102:0/704234698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f92c00111e0 con 0x564f154f7560 2024-04-24T23:24:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.482+0000 7f92dd661740 1 -- 172.21.15.102:0/704234698 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92c000eab0 con 0x564f154fbb60 2024-04-24T23:24:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.482+0000 7f92dd661740 1 -- 172.21.15.102:0/704234698 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92c000c920 con 0x564f154fb240 2024-04-24T23:24:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.482+0000 7f92d37fe640 1 -- 172.21.15.102:0/704234698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 msgr2=0x564f15503a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.482+0000 7f92d37fe640 1 --2- 172.21.15.102:0/704234698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15503a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.482+0000 7f92d37fe640 1 -- 172.21.15.102:0/704234698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154f7560 msgr2=0x564f15511f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.482+0000 7f92d37fe640 1 --2- 172.21.15.102:0/704234698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154f7560 0x564f15511f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.482+0000 7f92d37fe640 1 -- 172.21.15.102:0/704234698 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f15502130 con 0x564f154fb240 2024-04-24T23:24:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.483+0000 7f92d37fe640 1 --2- 172.21.15.102:0/704234698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fb240 0x564f15501a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92d411d3f0 tx=0x7f92c8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82a6e8c8ad603504 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.483+0000 7f92b9ffb640 1 -- 172.21.15.102:0/704234698 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c801ccc0 con 0x564f154fb240 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.484+0000 7f92b9ffb640 1 -- 172.21.15.102:0/704234698 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92c8025850 con 0x564f154fb240 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.484+0000 7f92b9ffb640 1 -- 172.21.15.102:0/704234698 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8010bb0 con 0x564f154fb240 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.484+0000 7f92dd661740 1 -- 172.21.15.102:0/704234698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fb240 msgr2=0x564f15501a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.484+0000 7f92dd661740 1 --2- 172.21.15.102:0/704234698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fb240 0x564f15501a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92d411d3f0 tx=0x7f92c8016000 comp rx=0 tx=0).stop 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.485+0000 7f92dd661740 1 -- 172.21.15.102:0/704234698 shutdown_connections 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.485+0000 7f92dd661740 1 --2- 172.21.15.102:0/704234698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154fbb60 0x564f15503a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.485+0000 7f92dd661740 1 --2- 172.21.15.102:0/704234698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fb240 0x564f15501a50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.485+0000 7f92dd661740 1 --2- 172.21.15.102:0/704234698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154f7560 0x564f15511f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.485+0000 7f92dd661740 1 -- 172.21.15.102:0/704234698 >> 172.21.15.102:0/704234698 conn(0x564f154feb50 msgr2=0x564f154f65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.485+0000 7f92dd661740 1 -- 172.21.15.102:0/704234698 shutdown_connections 2024-04-24T23:24:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.485+0000 7f92dd661740 1 -- 172.21.15.102:0/704234698 wait complete. 2024-04-24T23:24:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.485+0000 7f92dd661740 1 -- start start 2024-04-24T23:24:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.485+0000 7f92dd661740 1 Processor -- start 2024-04-24T23:24:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92dd661740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154f7560 0x564f1550e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92dd661740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fb240 0x564f15509cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92dd661740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fbb60 0x564f1550a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92dd661740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f92c000cf70 con 0x564f154fb240 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92dd661740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92c000e3e0 con 0x564f154f7560 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92dd661740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92c000d640 con 0x564f154fbb60 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92dd661740 10 client.0 _ll_register_callbacks cb 0x7f92cc004950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154f7560 0x564f1550e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154f7560 0x564f1550e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:58058/0 (socket says 172.21.15.102:58058) 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92d3fff640 1 -- 172.21.15.102:0/1179252194 learned_addr learned my addr 172.21.15.102:0/1179252194 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92d8ef7640 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fbb60 0x564f1550a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.486+0000 7f92d37fe640 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fb240 0x564f15509cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.487+0000 7f92d3fff640 1 -- 172.21.15.102:0/1179252194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fbb60 msgr2=0x564f1550a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.487+0000 7f92d3fff640 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fbb60 0x564f1550a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.487+0000 7f92d3fff640 1 -- 172.21.15.102:0/1179252194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fb240 msgr2=0x564f15509cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.487+0000 7f92d3fff640 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fb240 0x564f15509cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.487+0000 7f92d3fff640 1 -- 172.21.15.102:0/1179252194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f1550ab10 con 0x564f154f7560 2024-04-24T23:24:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.487+0000 7f92d3fff640 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154f7560 0x564f1550e960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f92cc031c20 tx=0x7f92cc0bd430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475210501c12f6e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.487+0000 7f92b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:24:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.489+0000 7f92b37fe640 1 -- 172.21.15.102:0/1179252194 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc02c040 con 0x564f154f7560 2024-04-24T23:24:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.489+0000 7f92b37fe640 1 -- 172.21.15.102:0/1179252194 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8579+0+0 (secure 0 0 0) 0x7f92cc0bcc10 con 0x564f154f7560 2024-04-24T23:24:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.489+0000 7f92dd661740 1 -- 172.21.15.102:0/1179252194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c8025850 con 0x564f154f7560 2024-04-24T23:24:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.490+0000 7f92b37fe640 1 -- 172.21.15.102:0/1179252194 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f92cc0dc1a0 con 0x564f154f7560 2024-04-24T23:24:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.491+0000 7f92dd661740 10 client.25721 fetch_fsmap learned FSMap version 26 2024-04-24T23:24:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.491+0000 7f92dd661740 1 -- 172.21.15.102:0/1179252194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f1558ddd0 con 0x564f154f7560 2024-04-24T23:24:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.491+0000 7f92b37fe640 1 -- 172.21.15.102:0/1179252194 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92c8025850 con 0x564f154f7560 2024-04-24T23:24:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.491+0000 7f92b37fe640 1 -- 172.21.15.102:0/1179252194 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f92cc0d1ce0 con 0x564f154f7560 2024-04-24T23:24:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.491+0000 7f92dd661740 10 client.25721 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:24:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.491+0000 7f92dd661740 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:24:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.491+0000 7f92dd661740 20 client.25721 populate_metadata read hostname 'smithi102' 2024-04-24T23:24:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.492+0000 7f92dd661740 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x564f15506230 0x564f155903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.492+0000 7f92dd661740 4 client.25721 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:24:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.492+0000 7f92dd661740 1 -- 172.21.15.102:0/1179252194 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f15467540 con 0x564f15506230 2024-04-24T23:24:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.493+0000 7f92d37fe640 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x564f15506230 0x564f155903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.493+0000 7f92d37fe640 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x564f15506230 0x564f155903d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def8139a65c12f5c server_cookie=6b9a126ef1fd2d3 in_seq=0 out_seq=0 2024-04-24T23:24:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.493+0000 7f92b37fe640 10 client.25721 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:57.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:57 smithi183 ceph-mon[33137]: pgmap v370: 97 pgs: 97 active+clean; 17 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:24:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.515+0000 7f92b37fe640 1 -- 172.21.15.102:0/1179252194 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f15467540 con 0x564f15506230 2024-04-24T23:24:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.515+0000 7f92b37fe640 10 client.25721 handle_command_reply: tid=0 2024-04-24T23:24:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.515+0000 7f92dd661740 1 client.25721 shutdown 2024-04-24T23:24:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92dd661740 1 -- 172.21.15.102:0/1179252194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154f7560 msgr2=0x564f1550e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92dd661740 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154f7560 0x564f1550e960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f92cc031c20 tx=0x7f92cc0bd430 comp rx=0 tx=0).stop 2024-04-24T23:24:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92dd661740 1 -- 172.21.15.102:0/1179252194 shutdown_connections 2024-04-24T23:24:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92dd661740 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f154fbb60 0x564f1550a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92dd661740 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x564f15506230 0x564f155903d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92b37fe640 0 client.25721 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:24:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92dd661740 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f154fb240 0x564f15509cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92dd661740 1 --2- 172.21.15.102:0/1179252194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f154f7560 0x564f1550e960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92dd661740 1 -- 172.21.15.102:0/1179252194 >> 172.21.15.102:0/1179252194 conn(0x564f154feb50 msgr2=0x564f154f65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.516+0000 7f92dd661740 1 -- 172.21.15.102:0/1179252194 shutdown_connections 2024-04-24T23:24:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.517+0000 7f92dd661740 1 -- 172.21.15.102:0/1179252194 wait complete. 2024-04-24T23:24:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:24:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:24:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:24:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:24:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.517+0000 7f92dd661740 20 client.25721 trim_cache size 0 max 16384 2024-04-24T23:24:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 -- 172.21.15.102:0/2578717994 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92bc0a2ea0 msgr2=0x7f92bc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92bc0a2ea0 0x7f92bc0a5360 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f92c80058c0 tx=0x7f92c8005850 comp rx=0 tx=0).stop 2024-04-24T23:24:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 -- 172.21.15.102:0/2578717994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 msgr2=0x7f92d41a6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 0x7f92d41a6da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005e30 tx=0x7f92cc003d60 comp rx=0 tx=0).stop 2024-04-24T23:24:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 -- 172.21.15.102:0/2578717994 shutdown_connections 2024-04-24T23:24:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92d41ab660 0x7f92d41abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f92bc0a2ea0 0x7f92bc0a5360 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92d411af20 0x7f92d41a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 --2- 172.21.15.102:0/2578717994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92d410ff30 0x7f92d41a6da0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 -- 172.21.15.102:0/2578717994 >> 172.21.15.102:0/2578717994 conn(0x7f92d406c810 msgr2=0x7f92d410dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 -- 172.21.15.102:0/2578717994 shutdown_connections 2024-04-24T23:24:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:24:57.518+0000 7f92da95a640 1 -- 172.21.15.102:0/2578717994 wait complete. 2024-04-24T23:24:59.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:24:59 smithi102 ceph-mon[29216]: pgmap v371: 97 pgs: 97 active+clean; 17 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 21 MiB/s wr, 5.44k op/s 2024-04-24T23:24:59.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:24:59 smithi161 ceph-mon[32709]: pgmap v371: 97 pgs: 97 active+clean; 17 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 21 MiB/s wr, 5.44k op/s 2024-04-24T23:24:59.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:24:59 smithi183 ceph-mon[33137]: pgmap v371: 97 pgs: 97 active+clean; 17 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 21 MiB/s wr, 5.44k op/s 2024-04-24T23:25:01.313 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:01 smithi183 ceph-mon[33137]: pgmap v372: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 28 MiB/s wr, 7.02k op/s 2024-04-24T23:25:01.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:01 smithi102 ceph-mon[29216]: pgmap v372: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 28 MiB/s wr, 7.02k op/s 2024-04-24T23:25:01.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:01 smithi161 ceph-mon[32709]: pgmap v372: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 28 MiB/s wr, 7.02k op/s 2024-04-24T23:25:03.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:03 smithi102 ceph-mon[29216]: pgmap v373: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 20 MiB/s wr, 5.07k op/s 2024-04-24T23:25:03.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:03 smithi161 ceph-mon[32709]: pgmap v373: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 20 MiB/s wr, 5.07k op/s 2024-04-24T23:25:03.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:03 smithi183 ceph-mon[33137]: pgmap v373: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 20 MiB/s wr, 5.07k op/s 2024-04-24T23:25:05.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:05 smithi102 ceph-mon[29216]: pgmap v374: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 27 MiB/s wr, 6.73k op/s 2024-04-24T23:25:05.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:05 smithi161 ceph-mon[32709]: pgmap v374: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 27 MiB/s wr, 6.73k op/s 2024-04-24T23:25:05.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:05 smithi183 ceph-mon[33137]: pgmap v374: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 27 MiB/s wr, 6.73k op/s 2024-04-24T23:25:07.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:07 smithi102 ceph-mon[29216]: pgmap v375: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 20 MiB/s wr, 5.08k op/s 2024-04-24T23:25:07.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:07 smithi102 ceph-mon[29216]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T23:25:07.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:07 smithi161 ceph-mon[32709]: pgmap v375: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 20 MiB/s wr, 5.08k op/s 2024-04-24T23:25:07.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:07 smithi161 ceph-mon[32709]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T23:25:07.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:07 smithi183 ceph-mon[33137]: pgmap v375: 97 pgs: 97 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 20 MiB/s wr, 5.08k op/s 2024-04-24T23:25:07.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:07 smithi183 ceph-mon[33137]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T23:25:09.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:09 smithi161 ceph-mon[32709]: pgmap v377: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 21 MiB/s wr, 5.37k op/s 2024-04-24T23:25:09.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:09 smithi183 ceph-mon[33137]: pgmap v377: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 21 MiB/s wr, 5.37k op/s 2024-04-24T23:25:09.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:09 smithi102 ceph-mon[29216]: pgmap v377: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 21 MiB/s wr, 5.37k op/s 2024-04-24T23:25:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.702+0000 7f6eed5ee640 1 -- 172.21.15.102:0/691337249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ee810e4b0 msgr2=0x7f6ee8108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.702+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/691337249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ee810e4b0 0x7f6ee8108740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009900 tx=0x7f6ed802a3f0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.703+0000 7f6eed5ee640 1 -- 172.21.15.102:0/691337249 shutdown_connections 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.703+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/691337249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ee810e4b0 0x7f6ee8108740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.703+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/691337249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee810da10 0x7f6ee810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.703+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/691337249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ee810fcb0 0x7f6ee8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.703+0000 7f6eed5ee640 1 -- 172.21.15.102:0/691337249 >> 172.21.15.102:0/691337249 conn(0x7f6ee806c6f0 msgr2=0x7f6ee806cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.703+0000 7f6eed5ee640 1 -- 172.21.15.102:0/691337249 shutdown_connections 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.703+0000 7f6eed5ee640 1 -- 172.21.15.102:0/691337249 wait complete. 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.704+0000 7f6eed5ee640 1 Processor -- start 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.704+0000 7f6eed5ee640 1 -- start start 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.704+0000 7f6eed5ee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ee810da10 0x7f6ee811e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.704+0000 7f6eed5ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee810e4b0 0x7f6ee811ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.704+0000 7f6eed5ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ee810fcb0 0x7f6ee8117720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.704+0000 7f6eed5ee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee8120610 con 0x7f6ee810da10 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.704+0000 7f6eed5ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee8120490 con 0x7f6ee810e4b0 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.704+0000 7f6eed5ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee8120790 con 0x7f6ee810fcb0 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee67fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee810e4b0 0x7f6ee811ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ee810fcb0 0x7f6ee8117720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ee810da10 0x7f6ee811e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ee810da10 0x7f6ee811e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58268/0 (socket says 172.21.15.102:58268) 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee6ffd640 1 -- 172.21.15.102:0/568787404 learned_addr learned my addr 172.21.15.102:0/568787404 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee6ffd640 1 -- 172.21.15.102:0/568787404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ee810fcb0 msgr2=0x7f6ee8117720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ee810fcb0 0x7f6ee8117720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee6ffd640 1 -- 172.21.15.102:0/568787404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee810e4b0 msgr2=0x7f6ee811ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee810e4b0 0x7f6ee811ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee6ffd640 1 -- 172.21.15.102:0/568787404 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee8117fe0 con 0x7f6ee810da10 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.705+0000 7f6ee77fe640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ee810fcb0 0x7f6ee8117720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.706+0000 7f6ee67fc640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee810e4b0 0x7f6ee811ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.706+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ee810da10 0x7f6ee811e4c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009d50 tx=0x7f6ee000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.707+0000 7f6ecffff640 1 -- 172.21.15.102:0/568787404 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0003b70 con 0x7f6ee810da10 2024-04-24T23:25:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.707+0000 7f6eed5ee640 1 -- 172.21.15.102:0/568787404 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee81182d0 con 0x7f6ee810da10 2024-04-24T23:25:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.707+0000 7f6eed5ee640 1 -- 172.21.15.102:0/568787404 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee811c2d0 con 0x7f6ee810da10 2024-04-24T23:25:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.707+0000 7f6ecffff640 1 -- 172.21.15.102:0/568787404 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ee0003d10 con 0x7f6ee810da10 2024-04-24T23:25:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.707+0000 7f6ecffff640 1 -- 172.21.15.102:0/568787404 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00108f0 con 0x7f6ee810da10 2024-04-24T23:25:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.711+0000 7f6ecffff640 1 -- 172.21.15.102:0/568787404 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f6ee0004000 con 0x7f6ee810da10 2024-04-24T23:25:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.715+0000 7f6ecffff640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6ed00a2f00 0x7f6ed00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.716+0000 7f6ecffff640 1 -- 172.21.15.102:0/568787404 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8623+0+0 (secure 0 0 0) 0x7f6ee00cf8e0 con 0x7f6ee810da10 2024-04-24T23:25:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.716+0000 7f6ef03d8740 1 Processor -- start 2024-04-24T23:25:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.716+0000 7f6ef03d8740 1 -- start start 2024-04-24T23:25:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.717+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 0x5600d1ab3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.717+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 0x5600d1abde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.717+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1abe3a0 0x5600d1ac0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.717+0000 7f6ef03d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5600d1a17cb0 con 0x5600d1ab33b0 2024-04-24T23:25:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.717+0000 7f6ef03d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5600d19c05b0 con 0x5600d1ab3cd0 2024-04-24T23:25:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.717+0000 7f6ef03d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5600d1a13c70 con 0x5600d1abe3a0 2024-04-24T23:25:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.717+0000 7f6ee6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 0x5600d1ab3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.717+0000 7f6ee6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 0x5600d1ab3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58282/0 (socket says 172.21.15.102:58282) 2024-04-24T23:25:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.717+0000 7f6ee6ffd640 1 -- 172.21.15.102:0/345486260 learned_addr learned my addr 172.21.15.102:0/345486260 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ee77fe640 1 --2- 172.21.15.102:0/345486260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1abe3a0 0x5600d1ac0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ee6ffd640 1 -- 172.21.15.102:0/345486260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1abe3a0 msgr2=0x5600d1ac0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/345486260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1abe3a0 0x5600d1ac0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ee6ffd640 1 -- 172.21.15.102:0/345486260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 msgr2=0x5600d1abde60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:25:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/345486260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 0x5600d1abde60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ee6ffd640 1 -- 172.21.15.102:0/345486260 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d172c210 con 0x5600d1ab33b0 2024-04-24T23:25:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/345486260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 0x5600d1ab3790 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00981a0 tx=0x7f6ee009b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4233c655a1acfee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ecd7fa640 1 -- 172.21.15.102:0/345486260 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00b9070 con 0x5600d1ab33b0 2024-04-24T23:25:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ecd7fa640 1 -- 172.21.15.102:0/345486260 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ee00a00f0 con 0x5600d1ab33b0 2024-04-24T23:25:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ecd7fa640 1 -- 172.21.15.102:0/345486260 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0097a90 con 0x5600d1ab33b0 2024-04-24T23:25:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.718+0000 7f6ee67fc640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6ed00a2f00 0x7f6ed00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.719+0000 7f6ef03d8740 1 -- 172.21.15.102:0/345486260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 msgr2=0x5600d1ab3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.720+0000 7f6ee67fc640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6ed00a2f00 0x7f6ed00a53c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5600d1ab4970 tx=0x7f6edc005f90 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.720+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/345486260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 0x5600d1ab3790 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00981a0 tx=0x7f6ee009b6a0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.720+0000 7f6ef03d8740 1 -- 172.21.15.102:0/345486260 shutdown_connections 2024-04-24T23:25:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.720+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/345486260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1abe3a0 0x5600d1ac0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.720+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/345486260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 0x5600d1abde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.720+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/345486260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 0x5600d1ab3790 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.720+0000 7f6ef03d8740 1 -- 172.21.15.102:0/345486260 >> 172.21.15.102:0/345486260 conn(0x5600d1aaf6d0 msgr2=0x5600d1aafab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.720+0000 7f6ef03d8740 1 -- 172.21.15.102:0/345486260 shutdown_connections 2024-04-24T23:25:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.720+0000 7f6ef03d8740 1 -- 172.21.15.102:0/345486260 wait complete. 2024-04-24T23:25:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ef03d8740 1 -- start start 2024-04-24T23:25:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ef03d8740 1 Processor -- start 2024-04-24T23:25:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab33b0 0x5600d1abb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab3cd0 0x5600d1abb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1abbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ef03d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5600d1ac1190 con 0x5600d1abe3a0 2024-04-24T23:25:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ef03d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5600d19d3b50 con 0x5600d1ab33b0 2024-04-24T23:25:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ef03d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5600d1ac1310 con 0x5600d1ab3cd0 2024-04-24T23:25:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ef03d8740 10 client.0 _ll_register_callbacks cb 0x5600d1a0eca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ee77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1abbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ee77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1abbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58296/0 (socket says 172.21.15.102:58296) 2024-04-24T23:25:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ee77fe640 1 -- 172.21.15.102:0/574720508 learned_addr learned my addr 172.21.15.102:0/574720508 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.721+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab33b0 0x5600d1abb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.722+0000 7f6ee77fe640 1 -- 172.21.15.102:0/574720508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab3cd0 msgr2=0x5600d1abb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.722+0000 7f6ee77fe640 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab3cd0 0x5600d1abb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.722+0000 7f6ee77fe640 1 -- 172.21.15.102:0/574720508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab33b0 msgr2=0x5600d1abb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.722+0000 7f6ee77fe640 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab33b0 0x5600d1abb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.722+0000 7f6ee77fe640 1 -- 172.21.15.102:0/574720508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d17f8620 con 0x5600d1abe3a0 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.722+0000 7f6ee77fe640 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1abbde0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6ee810e9e0 tx=0x7f6ed8005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd086a0bf2b6a127 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.722+0000 7f6ecaff5640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.723+0000 7f6ecaff5640 1 -- 172.21.15.102:0/574720508 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8035040 con 0x5600d1abe3a0 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.723+0000 7f6ecaff5640 1 -- 172.21.15.102:0/574720508 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8623+0+0 (secure 0 0 0) 0x7f6ed80048c0 con 0x5600d1abe3a0 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.723+0000 7f6ef03d8740 1 -- 172.21.15.102:0/574720508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee00a00f0 con 0x5600d1abe3a0 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.723+0000 7f6ecaff5640 1 -- 172.21.15.102:0/574720508 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ed8004ac0 con 0x5600d1abe3a0 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.723+0000 7f6ef03d8740 10 client.16311 fetch_fsmap learned FSMap version 26 2024-04-24T23:25:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.723+0000 7f6ef03d8740 1 -- 172.21.15.102:0/574720508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600d1acd220 con 0x5600d1abe3a0 2024-04-24T23:25:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.724+0000 7f6ecaff5640 1 -- 172.21.15.102:0/574720508 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ed8041030 con 0x5600d1abe3a0 2024-04-24T23:25:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.724+0000 7f6ecaff5640 1 -- 172.21.15.102:0/574720508 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6ed80314a0 con 0x5600d1abe3a0 2024-04-24T23:25:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.724+0000 7f6ef03d8740 10 client.16311 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:25:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.724+0000 7f6ef03d8740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:25:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.724+0000 7f6ef03d8740 20 client.16311 populate_metadata read hostname 'smithi102' 2024-04-24T23:25:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.724+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5600d1ace7a0 0x5600d1ad0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.724+0000 7f6ef03d8740 4 client.16311 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.724+0000 7f6ef03d8740 1 -- 172.21.15.102:0/574720508 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ee00a00f0 con 0x5600d1ace7a0 2024-04-24T23:25:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.727+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5600d1ace7a0 0x5600d1ad0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.728+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5600d1ace7a0 0x5600d1ad0b90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5d745c718f54ef server_cookie=73f183cf1a44c602 in_seq=0 out_seq=0 2024-04-24T23:25:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.728+0000 7f6ecaff5640 10 client.16311 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.728+0000 7f6ecaff5640 1 -- 172.21.15.102:0/574720508 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f6ee00a00f0 con 0x5600d1ace7a0 2024-04-24T23:25:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.728+0000 7f6ecaff5640 10 client.16311 handle_command_reply: tid=0 2024-04-24T23:25:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.728+0000 7f6ef03d8740 1 client.16311 shutdown 2024-04-24T23:25:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.732+0000 7f6ef03d8740 1 -- 172.21.15.102:0/574720508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 msgr2=0x5600d1abbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.732+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1abbde0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6ee810e9e0 tx=0x7f6ed8005270 comp rx=0 tx=0).stop 2024-04-24T23:25:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.734+0000 7f6ef03d8740 1 -- 172.21.15.102:0/574720508 shutdown_connections 2024-04-24T23:25:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.734+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5600d1ace7a0 0x5600d1ad0b90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.734+0000 7f6ecaff5640 0 client.16311 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.734+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1abbde0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.734+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab3cd0 0x5600d1abb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.734+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/574720508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab33b0 0x5600d1abb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.734+0000 7f6ef03d8740 1 -- 172.21.15.102:0/574720508 >> 172.21.15.102:0/574720508 conn(0x5600d1aaf6d0 msgr2=0x5600d1ab53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.735+0000 7f6ef03d8740 1 -- 172.21.15.102:0/574720508 shutdown_connections 2024-04-24T23:25:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.735+0000 7f6ef03d8740 1 -- 172.21.15.102:0/574720508 wait complete. 2024-04-24T23:25:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.736+0000 7f6ef03d8740 20 client.16311 trim_cache size 0 max 16384 2024-04-24T23:25:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.772+0000 7f6ef03d8740 1 Processor -- start 2024-04-24T23:25:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.774+0000 7f6ef03d8740 1 -- start start 2024-04-24T23:25:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.774+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 0x5600d1aca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.774+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab3cd0 0x5600d1abbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.774+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1abe3a0 0x5600d1ad6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ee77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1abe3a0 0x5600d1ad6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ee77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1abe3a0 0x5600d1ad6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:38986/0 (socket says 172.21.15.102:38986) 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ee77fe640 1 -- 172.21.15.102:0/2283591062 learned_addr learned my addr 172.21.15.102:0/2283591062 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ef03d8740 1 -- 172.21.15.102:0/2283591062 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed40112c0 con 0x5600d1ab33b0 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ef03d8740 1 -- 172.21.15.102:0/2283591062 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed400eb90 con 0x5600d1abe3a0 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ef03d8740 1 -- 172.21.15.102:0/2283591062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed400ca00 con 0x5600d1ab3cd0 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ee77fe640 1 -- 172.21.15.102:0/2283591062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab3cd0 msgr2=0x5600d1abbc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ee77fe640 1 --2- 172.21.15.102:0/2283591062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab3cd0 0x5600d1abbc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ee77fe640 1 -- 172.21.15.102:0/2283591062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 msgr2=0x5600d1aca080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ee77fe640 1 --2- 172.21.15.102:0/2283591062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 0x5600d1aca080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.775+0000 7f6ee77fe640 1 -- 172.21.15.102:0/2283591062 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d1aaeea0 con 0x5600d1abe3a0 2024-04-24T23:25:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.776+0000 7f6ee77fe640 1 --2- 172.21.15.102:0/2283591062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1abe3a0 0x5600d1ad6480 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8031ad0 tx=0x7f6ed8004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c881c6e9de509bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.776+0000 7f6eccff9640 1 -- 172.21.15.102:0/2283591062 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8052020 con 0x5600d1abe3a0 2024-04-24T23:25:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.777+0000 7f6eccff9640 1 -- 172.21.15.102:0/2283591062 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ed8041070 con 0x5600d1abe3a0 2024-04-24T23:25:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.777+0000 7f6eccff9640 1 -- 172.21.15.102:0/2283591062 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8035070 con 0x5600d1abe3a0 2024-04-24T23:25:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.778+0000 7f6ef03d8740 1 -- 172.21.15.102:0/2283591062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1abe3a0 msgr2=0x5600d1ad6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.778+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/2283591062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1abe3a0 0x5600d1ad6480 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8031ad0 tx=0x7f6ed8004610 comp rx=0 tx=0).stop 2024-04-24T23:25:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.778+0000 7f6ef03d8740 1 -- 172.21.15.102:0/2283591062 shutdown_connections 2024-04-24T23:25:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.778+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/2283591062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1abe3a0 0x5600d1ad6480 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.778+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/2283591062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab3cd0 0x5600d1abbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.778+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/2283591062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1ab33b0 0x5600d1aca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.779+0000 7f6ef03d8740 1 -- 172.21.15.102:0/2283591062 >> 172.21.15.102:0/2283591062 conn(0x5600d1aaf6d0 msgr2=0x5600d1abca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.779+0000 7f6ef03d8740 1 -- 172.21.15.102:0/2283591062 shutdown_connections 2024-04-24T23:25:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.779+0000 7f6ef03d8740 1 -- 172.21.15.102:0/2283591062 wait complete. 2024-04-24T23:25:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.780+0000 7f6ef03d8740 1 -- start start 2024-04-24T23:25:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.780+0000 7f6ef03d8740 1 Processor -- start 2024-04-24T23:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.780+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab33b0 0x5600d1ac6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.780+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 0x5600d1ac6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.780+0000 7f6ef03d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1ac35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.780+0000 7f6ee67fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 0x5600d1ac6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.780+0000 7f6ee6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab33b0 0x5600d1ac6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.780+0000 7f6ee6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab33b0 0x5600d1ac6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:50752/0 (socket says 172.21.15.102:50752) 2024-04-24T23:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.780+0000 7f6ee67fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 0x5600d1ac6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:38988/0 (socket says 172.21.15.102:38988) 2024-04-24T23:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ee77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1ac35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ee77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1ac35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58312/0 (socket says 172.21.15.102:58312) 2024-04-24T23:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ee6ffd640 1 -- 172.21.15.102:0/3453486301 learned_addr learned my addr 172.21.15.102:0/3453486301 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed400d050 con 0x5600d1abe3a0 2024-04-24T23:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed400e4c0 con 0x5600d1ab3cd0 2024-04-24T23:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed400d720 con 0x5600d1ab33b0 2024-04-24T23:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ef03d8740 10 client.0 _ll_register_callbacks cb 0x7f6edc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ee67fc640 1 -- 172.21.15.102:0/3453486301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab33b0 msgr2=0x5600d1ac6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ee67fc640 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab33b0 0x5600d1ac6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ee67fc640 1 -- 172.21.15.102:0/3453486301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 msgr2=0x5600d1ac35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ee67fc640 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1ac35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.781+0000 7f6ee67fc640 1 -- 172.21.15.102:0/3453486301 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d1ac3df0 con 0x5600d1ab3cd0 2024-04-24T23:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.782+0000 7f6ee67fc640 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 0x5600d1ac6ad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5600d1abc930 tx=0x7f6edc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca69cb82a5c2649 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.782+0000 7f6ecaff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.783+0000 7f6ecaff5640 1 -- 172.21.15.102:0/3453486301 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc003e50 con 0x5600d1ab3cd0 2024-04-24T23:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.783+0000 7f6ecaff5640 1 -- 172.21.15.102:0/3453486301 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8623+0+0 (secure 0 0 0) 0x7f6edc012450 con 0x5600d1ab3cd0 2024-04-24T23:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.783+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ed8041070 con 0x5600d1ab3cd0 2024-04-24T23:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.783+0000 7f6ecaff5640 1 -- 172.21.15.102:0/3453486301 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6edc012650 con 0x5600d1ab3cd0 2024-04-24T23:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.784+0000 7f6ef03d8740 10 client.25736 fetch_fsmap learned FSMap version 26 2024-04-24T23:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.784+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600d1b358f0 con 0x5600d1ab3cd0 2024-04-24T23:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.784+0000 7f6ecaff5640 1 -- 172.21.15.102:0/3453486301 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6edc022030 con 0x5600d1ab3cd0 2024-04-24T23:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.784+0000 7f6ecaff5640 1 -- 172.21.15.102:0/3453486301 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6edc031840 con 0x5600d1ab3cd0 2024-04-24T23:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.784+0000 7f6ef03d8740 10 client.25736 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.784+0000 7f6ef03d8740 10 client.25736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.785+0000 7f6ef03d8740 20 client.25736 populate_metadata read hostname 'smithi102' 2024-04-24T23:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.785+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5600d1ace7a0 0x5600d1b38050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.785+0000 7f6ef03d8740 4 client.25736 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:25:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.785+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6ed8041070 con 0x5600d1ace7a0 2024-04-24T23:25:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.785+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5600d1ace7a0 0x5600d1b38050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.786+0000 7f6ee6ffd640 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5600d1ace7a0 0x5600d1b38050 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e7b0b4c2660e85 server_cookie=4c7148c210fba316 in_seq=0 out_seq=0 2024-04-24T23:25:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.786+0000 7f6ecaff5640 10 client.25736 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.796+0000 7f6ecaff5640 1 -- 172.21.15.102:0/3453486301 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6ed8041070 con 0x5600d1ace7a0 2024-04-24T23:25:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.796+0000 7f6ecaff5640 10 client.25736 handle_command_reply: tid=0 2024-04-24T23:25:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.797+0000 7f6ef03d8740 1 client.25736 shutdown 2024-04-24T23:25:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.801+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 msgr2=0x5600d1ac6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.801+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 0x5600d1ac6ad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5600d1abc930 tx=0x7f6edc0046c0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.801+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 shutdown_connections 2024-04-24T23:25:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.801+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x5600d1ace7a0 0x5600d1b38050 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.802+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5600d1abe3a0 0x5600d1ac35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.802+0000 7f6ecaff5640 0 client.25736 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.802+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600d1ab3cd0 0x5600d1ac6ad0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.802+0000 7f6ef03d8740 1 --2- 172.21.15.102:0/3453486301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600d1ab33b0 0x5600d1ac6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.802+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 >> 172.21.15.102:0/3453486301 conn(0x5600d1aaf6d0 msgr2=0x5600d1abb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.802+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 shutdown_connections 2024-04-24T23:25:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.802+0000 7f6ef03d8740 1 -- 172.21.15.102:0/3453486301 wait complete. 2024-04-24T23:25:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.802+0000 7f6ef03d8740 20 client.25736 trim_cache size 0 max 16384 2024-04-24T23:25:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:25:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:25:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:25:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:25:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.803+0000 7f6eed5ee640 1 -- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6ed00a2f00 msgr2=0x7f6ed00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.803+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6ed00a2f00 0x7f6ed00a53c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5600d1ab4970 tx=0x7f6edc005f90 comp rx=0 tx=0).stop 2024-04-24T23:25:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.804+0000 7f6eed5ee640 1 -- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ee810da10 msgr2=0x7f6ee811e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.804+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ee810da10 0x7f6ee811e4c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009d50 tx=0x7f6ee000c6a0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.804+0000 7f6eed5ee640 1 -- 172.21.15.102:0/568787404 shutdown_connections 2024-04-24T23:25:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.804+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ee810fcb0 0x7f6ee8117720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.805+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee810e4b0 0x7f6ee811ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.805+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f6ed00a2f00 0x7f6ed00a53c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.805+0000 7f6eed5ee640 1 --2- 172.21.15.102:0/568787404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ee810da10 0x7f6ee811e4c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.805+0000 7f6eed5ee640 1 -- 172.21.15.102:0/568787404 >> 172.21.15.102:0/568787404 conn(0x7f6ee806c6f0 msgr2=0x7f6ee8109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.805+0000 7f6eed5ee640 1 -- 172.21.15.102:0/568787404 shutdown_connections 2024-04-24T23:25:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:10.805+0000 7f6eed5ee640 1 -- 172.21.15.102:0/568787404 wait complete. 2024-04-24T23:25:11.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:11 smithi161 ceph-mon[32709]: pgmap v378: 97 pgs: 11 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 67 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 18 MiB/s wr, 4.67k op/s 2024-04-24T23:25:11.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:11 smithi183 ceph-mon[33137]: pgmap v378: 97 pgs: 11 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 67 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 18 MiB/s wr, 4.67k op/s 2024-04-24T23:25:11.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:11 smithi102 ceph-mon[29216]: pgmap v378: 97 pgs: 11 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 67 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 18 MiB/s wr, 4.67k op/s 2024-04-24T23:25:13.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:13 smithi161 ceph-mon[32709]: pgmap v379: 97 pgs: 11 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 67 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 20 MiB/s wr, 5.00k op/s 2024-04-24T23:25:13.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:13 smithi183 ceph-mon[33137]: pgmap v379: 97 pgs: 11 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 67 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 20 MiB/s wr, 5.00k op/s 2024-04-24T23:25:13.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:13 smithi102 ceph-mon[29216]: pgmap v379: 97 pgs: 11 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 67 active+clean; 17 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 20 MiB/s wr, 5.00k op/s 2024-04-24T23:25:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:15 smithi161 ceph-mon[32709]: pgmap v380: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 20 MiB/s wr, 5.12k op/s 2024-04-24T23:25:15.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:15 smithi161 ceph-mon[32709]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T23:25:15.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:15 smithi183 ceph-mon[33137]: pgmap v380: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 20 MiB/s wr, 5.12k op/s 2024-04-24T23:25:15.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:15 smithi183 ceph-mon[33137]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T23:25:15.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:15 smithi102 ceph-mon[29216]: pgmap v380: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 20 MiB/s wr, 5.12k op/s 2024-04-24T23:25:15.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:15 smithi102 ceph-mon[29216]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T23:25:17.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:17 smithi161 ceph-mon[32709]: pgmap v382: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:25:17.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:17 smithi183 ceph-mon[33137]: pgmap v382: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:25:17.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:17 smithi102 ceph-mon[29216]: pgmap v382: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:25:19.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:19 smithi161 ceph-mon[32709]: pgmap v383: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 21 MiB/s wr, 5.30k op/s 2024-04-24T23:25:19.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:19 smithi183 ceph-mon[33137]: pgmap v383: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 21 MiB/s wr, 5.30k op/s 2024-04-24T23:25:19.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:19 smithi102 ceph-mon[29216]: pgmap v383: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 21 MiB/s wr, 5.30k op/s 2024-04-24T23:25:21.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:21 smithi183 ceph-mon[33137]: pgmap v384: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 21 MiB/s wr, 5.33k op/s 2024-04-24T23:25:21.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:21 smithi102 ceph-mon[29216]: pgmap v384: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 21 MiB/s wr, 5.33k op/s 2024-04-24T23:25:21.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:21 smithi161 ceph-mon[32709]: pgmap v384: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 21 MiB/s wr, 5.33k op/s 2024-04-24T23:25:23.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:23 smithi183 ceph-mon[33137]: pgmap v385: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 21 MiB/s wr, 5.36k op/s 2024-04-24T23:25:23.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:23 smithi102 ceph-mon[29216]: pgmap v385: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 21 MiB/s wr, 5.36k op/s 2024-04-24T23:25:23.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:23 smithi161 ceph-mon[32709]: pgmap v385: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 21 MiB/s wr, 5.36k op/s 2024-04-24T23:25:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 -- 172.21.15.102:0/3271520203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094106650 msgr2=0x7ff094106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/3271520203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094106650 0x7ff094106a30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff090009930 tx=0x7ff09002a420 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 -- 172.21.15.102:0/3271520203 shutdown_connections 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/3271520203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff094101000 0x7ff09410d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/3271520203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff094100640 0x7ff094100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/3271520203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094106650 0x7ff094106a30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 -- 172.21.15.102:0/3271520203 >> 172.21.15.102:0/3271520203 conn(0x7ff094077760 msgr2=0x7ff094077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 -- 172.21.15.102:0/3271520203 shutdown_connections 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 -- 172.21.15.102:0/3271520203 wait complete. 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 Processor -- start 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.960+0000 7ff09c7c8640 1 -- start start 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09c7c8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094100640 0x7ff094196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09c7c8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff094101000 0x7ff094197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09c7c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff094106650 0x7ff09419b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09c7c8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff094110920 con 0x7ff094100640 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09c7c8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941107a0 con 0x7ff094106650 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09c7c8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff094110aa0 con 0x7ff094101000 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09a564640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094100640 0x7ff094196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09a564640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094100640 0x7ff094196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59120/0 (socket says 172.21.15.102:59120) 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09a564640 1 -- 172.21.15.102:0/2170102952 learned_addr learned my addr 172.21.15.102:0/2170102952 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09ad65640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff094106650 0x7ff09419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff099d63640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff094101000 0x7ff094197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09a564640 1 -- 172.21.15.102:0/2170102952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff094101000 msgr2=0x7ff094197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09a564640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff094101000 0x7ff094197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09a564640 1 -- 172.21.15.102:0/2170102952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff094106650 msgr2=0x7ff09419b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09a564640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff094106650 0x7ff09419b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09a564640 1 -- 172.21.15.102:0/2170102952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09419be00 con 0x7ff094100640 2024-04-24T23:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09a564640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094100640 0x7ff094196e40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff090005ab0 tx=0x7ff090003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff08b7fe640 1 -- 172.21.15.102:0/2170102952 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09003f070 con 0x7ff094100640 2024-04-24T23:25:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff08b7fe640 1 -- 172.21.15.102:0/2170102952 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff090031d60 con 0x7ff094100640 2024-04-24T23:25:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.961+0000 7ff09c7c8640 1 -- 172.21.15.102:0/2170102952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09419c090 con 0x7ff094100640 2024-04-24T23:25:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.962+0000 7ff08b7fe640 1 -- 172.21.15.102:0/2170102952 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090035030 con 0x7ff094100640 2024-04-24T23:25:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.962+0000 7ff09c7c8640 1 -- 172.21.15.102:0/2170102952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0940699e0 con 0x7ff094100640 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.964+0000 7ff08b7fe640 1 -- 172.21.15.102:0/2170102952 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff090042070 con 0x7ff094100640 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.964+0000 7ff09f5b1740 1 Processor -- start 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09f5b1740 1 -- start start 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c7f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3c89cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c8a230 0x558be3c8c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558be3be3cb0 con 0x558be3c8a230 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558be3b8c5b0 con 0x558be3c7fb60 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558be3bdfc70 con 0x558be3c7f240 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff08b7fe640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0800a2da0 0x7ff0800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff099d63640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c8a230 0x558be3c8c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff099d63640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c8a230 0x558be3c8c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59126/0 (socket says 172.21.15.102:59126) 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff099d63640 1 -- 172.21.15.102:0/3567187748 learned_addr learned my addr 172.21.15.102:0/3567187748 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09ad65640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0800a2da0 0x7ff0800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09a564640 1 --2- 172.21.15.102:0/3567187748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3c89cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff09ad65640 1 --2- 172.21.15.102:0/3567187748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c7f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff099d63640 1 -- 172.21.15.102:0/3567187748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 msgr2=0x558be3c7f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff099d63640 1 --2- 172.21.15.102:0/3567187748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c7f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff099d63640 1 -- 172.21.15.102:0/3567187748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 msgr2=0x558be3c89cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff099d63640 1 --2- 172.21.15.102:0/3567187748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3c89cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff099d63640 1 -- 172.21.15.102:0/3567187748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be38f8210 con 0x558be3c8a230 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff08b7fe640 1 -- 172.21.15.102:0/2170102952 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7ff0900efb10 con 0x7ff094100640 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.965+0000 7ff099d63640 1 --2- 172.21.15.102:0/3567187748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c8a230 0x558be3c8c620 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff08400bc80 tx=0x7ff08400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2389954aad2aeec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09ad65640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0800a2da0 0x7ff0800a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009fd0 tx=0x7ff08c009ad0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff088ff9640 1 -- 172.21.15.102:0/3567187748 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084015070 con 0x558be3c8a230 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff088ff9640 1 -- 172.21.15.102:0/3567187748 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0840118f0 con 0x558be3c8a230 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff088ff9640 1 -- 172.21.15.102:0/3567187748 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084010830 con 0x558be3c8a230 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09f5b1740 1 -- 172.21.15.102:0/3567187748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c8a230 msgr2=0x558be3c8c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/3567187748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c8a230 0x558be3c8c620 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff08400bc80 tx=0x7ff08400f730 comp rx=0 tx=0).stop 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09f5b1740 1 -- 172.21.15.102:0/3567187748 shutdown_connections 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/3567187748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c8a230 0x558be3c8c620 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/3567187748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3c89cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/3567187748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c7f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09f5b1740 1 -- 172.21.15.102:0/3567187748 >> 172.21.15.102:0/3567187748 conn(0x558be3c7b560 msgr2=0x558be3c7b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09f5b1740 1 -- 172.21.15.102:0/3567187748 shutdown_connections 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.966+0000 7ff09f5b1740 1 -- 172.21.15.102:0/3567187748 wait complete. 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09f5b1740 1 -- start start 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09f5b1740 1 Processor -- start 2024-04-24T23:25:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c7f240 0x558be3c87300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7fb60 0x558be3c87840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c87d80 0x558be3c96490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558be3c8d020 con 0x558be3c7f240 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09a564640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c7f240 0x558be3c87300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558be3b9fb50 con 0x558be3c87d80 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558be3c8d1a0 con 0x558be3c7fb60 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09a564640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c7f240 0x558be3c87300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59132/0 (socket says 172.21.15.102:59132) 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09a564640 1 -- 172.21.15.102:0/218317005 learned_addr learned my addr 172.21.15.102:0/218317005 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09f5b1740 10 client.0 _ll_register_callbacks cb 0x558be3bdaca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09ad65640 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c87d80 0x558be3c96490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09a564640 1 -- 172.21.15.102:0/218317005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7fb60 msgr2=0x558be3c87840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09a564640 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7fb60 0x558be3c87840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09a564640 1 -- 172.21.15.102:0/218317005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c87d80 msgr2=0x558be3c96490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09a564640 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c87d80 0x558be3c96490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09a564640 1 -- 172.21.15.102:0/218317005 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be39c4620 con 0x558be3c7f240 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.967+0000 7ff09ad65640 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c87d80 0x558be3c96490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09f5b1740 1 -- 172.21.15.102:0/218317005 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0840118f0 con 0x558be3c7f240 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09a564640 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c7f240 0x558be3c87300 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff0900b89c0 tx=0x7ff09008af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde63e04f651046 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff07dffb640 10 client.16326 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff07dffb640 1 -- 172.21.15.102:0/218317005 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090049070 con 0x558be3c7f240 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff07dffb640 1 -- 172.21.15.102:0/218317005 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7ff0900a5170 con 0x558be3c7f240 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff07dffb640 1 -- 172.21.15.102:0/218317005 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0900dc0b0 con 0x558be3c7f240 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09f5b1740 10 client.16326 fetch_fsmap learned FSMap version 26 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09f5b1740 1 -- 172.21.15.102:0/218317005 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=26}) -- 0x558be3c99720 con 0x558be3c7f240 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff07dffb640 1 -- 172.21.15.102:0/218317005 <== mon.0 v2:172.21.15.102:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff09003b580 con 0x558be3c7f240 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09f5b1740 10 client.16326 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09f5b1740 10 client.16326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09f5b1740 20 client.16326 populate_metadata read hostname 'smithi102' 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558be3c9ab30 0x558be3c9cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09f5b1740 4 client.16326 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff09f5b1740 1 -- 172.21.15.102:0/218317005 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be3bf11f0 con 0x558be3c9ab30 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.968+0000 7ff099d63640 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558be3c9ab30 0x558be3c9cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff07dffb640 10 client.16326 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff099d63640 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558be3c9ab30 0x558be3c9cff0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c0f5a1337c715a server_cookie=72c8ef2280758575 in_seq=0 out_seq=0 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff07dffb640 1 -- 172.21.15.102:0/218317005 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558be3bf11f0 con 0x558be3c9ab30 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff07dffb640 10 client.16326 handle_command_reply: tid=0 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff09f5b1740 1 client.16326 shutdown 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff09f5b1740 1 -- 172.21.15.102:0/218317005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c7f240 msgr2=0x558be3c87300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c7f240 0x558be3c87300 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff0900b89c0 tx=0x7ff09008af20 comp rx=0 tx=0).stop 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff09f5b1740 1 -- 172.21.15.102:0/218317005 shutdown_connections 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c87d80 0x558be3c96490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7fb60 0x558be3c87840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558be3c9ab30 0x558be3c9cff0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/218317005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c7f240 0x558be3c87300 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff07dffb640 0 client.16326 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.969+0000 7ff09f5b1740 1 -- 172.21.15.102:0/218317005 >> 172.21.15.102:0/218317005 conn(0x558be3c7b560 msgr2=0x558be3c81360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.970+0000 7ff09f5b1740 1 -- 172.21.15.102:0/218317005 shutdown_connections 2024-04-24T23:25:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.970+0000 7ff09f5b1740 1 -- 172.21.15.102:0/218317005 wait complete. 2024-04-24T23:25:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.970+0000 7ff09f5b1740 20 client.16326 trim_cache size 0 max 16384 2024-04-24T23:25:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09f5b1740 1 Processor -- start 2024-04-24T23:25:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09f5b1740 1 -- start start 2024-04-24T23:25:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c7a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3ca2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3ca2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941b1e80 con 0x558be3c87d80 2024-04-24T23:25:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941af750 con 0x558be3c7fb60 2024-04-24T23:25:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941ad5c0 con 0x558be3c7f240 2024-04-24T23:25:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09ad65640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3ca2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09ad65640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3ca2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59144/0 (socket says 172.21.15.102:59144) 2024-04-24T23:25:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09ad65640 1 -- 172.21.15.102:0/1271355723 learned_addr learned my addr 172.21.15.102:0/1271355723 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff099d63640 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3ca2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09a564640 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c7a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09ad65640 1 -- 172.21.15.102:0/1271355723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 msgr2=0x558be3c7a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09ad65640 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c7a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09ad65640 1 -- 172.21.15.102:0/1271355723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 msgr2=0x558be3ca2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09ad65640 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3ca2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09ad65640 1 -- 172.21.15.102:0/1271355723 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be3c8c990 con 0x558be3c87d80 2024-04-24T23:25:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff09a564640 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c7a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.987+0000 7ff099d63640 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3ca2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09ad65640 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3ca2e40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009fa0 tx=0x7ff08c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658fa9cd5f6cacad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff07ffff640 1 -- 172.21.15.102:0/1271355723 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c02d070 con 0x558be3c87d80 2024-04-24T23:25:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff07ffff640 1 -- 172.21.15.102:0/1271355723 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff08c020b60 con 0x558be3c87d80 2024-04-24T23:25:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff07ffff640 1 -- 172.21.15.102:0/1271355723 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0283f0 con 0x558be3c87d80 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09f5b1740 1 -- 172.21.15.102:0/1271355723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 msgr2=0x558be3ca2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3ca2e40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009fa0 tx=0x7ff08c0096a0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09f5b1740 1 -- 172.21.15.102:0/1271355723 shutdown_connections 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3ca2e40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3ca2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/1271355723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c7a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09f5b1740 1 -- 172.21.15.102:0/1271355723 >> 172.21.15.102:0/1271355723 conn(0x558be3c7b560 msgr2=0x558be3c79d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09f5b1740 1 -- 172.21.15.102:0/1271355723 shutdown_connections 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.988+0000 7ff09f5b1740 1 -- 172.21.15.102:0/1271355723 wait complete. 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09f5b1740 1 -- start start 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09f5b1740 1 Processor -- start 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c84360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3c848a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09f5b1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3d00170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941adc10 con 0x558be3c87d80 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941af080 con 0x558be3c7fb60 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09f5b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0941ae2e0 con 0x558be3c7f240 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09f5b1740 10 client.0 _ll_register_callbacks cb 0x7ff09003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09ad65640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3d00170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09ad65640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3d00170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59156/0 (socket says 172.21.15.102:59156) 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09ad65640 1 -- 172.21.15.102:0/789918764 learned_addr learned my addr 172.21.15.102:0/789918764 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09a564640 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c84360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff099d63640 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3c848a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09ad65640 1 -- 172.21.15.102:0/789918764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 msgr2=0x558be3c84360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09ad65640 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c84360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09ad65640 1 -- 172.21.15.102:0/789918764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 msgr2=0x558be3c848a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09ad65640 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3c848a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09ad65640 1 -- 172.21.15.102:0/789918764 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be3d006b0 con 0x558be3c87d80 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09a564640 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c84360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff09ad65640 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3d00170 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff08c01ec30 tx=0x7ff08c010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f13a8f60b51fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.989+0000 7ff07dffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff07dffb640 1 -- 172.21.15.102:0/789918764 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c02d040 con 0x558be3c87d80 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff07dffb640 1 -- 172.21.15.102:0/789918764 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7ff08c020960 con 0x558be3c87d80 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09f5b1740 1 -- 172.21.15.102:0/789918764 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08c020b60 con 0x558be3c87d80 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff07dffb640 1 -- 172.21.15.102:0/789918764 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff08c01ed70 con 0x558be3c87d80 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09f5b1740 10 client.16332 fetch_fsmap learned FSMap version 26 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09f5b1740 1 -- 172.21.15.102:0/789918764 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558be3d01800 con 0x558be3c87d80 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff07dffb640 1 -- 172.21.15.102:0/789918764 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff08c020b60 con 0x558be3c87d80 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff07dffb640 1 -- 172.21.15.102:0/789918764 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff08c027030 con 0x558be3c87d80 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09f5b1740 10 client.16332 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09f5b1740 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09f5b1740 20 client.16332 populate_metadata read hostname 'smithi102' 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558be3d01e60 0x558be3d04250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09f5b1740 4 client.16332 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:25:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09f5b1740 1 -- 172.21.15.102:0/789918764 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558be3beb540 con 0x558be3d01e60 2024-04-24T23:25:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.990+0000 7ff09a564640 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558be3d01e60 0x558be3d04250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.991+0000 7ff07dffb640 10 client.16332 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.991+0000 7ff09a564640 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558be3d01e60 0x558be3d04250 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cef874d2f7a86b5 server_cookie=b72dfbfa7b3eceff in_seq=0 out_seq=0 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff07dffb640 1 -- 172.21.15.102:0/789918764 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558be3beb540 con 0x558be3d01e60 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff07dffb640 10 client.16332 handle_command_reply: tid=0 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 client.16332 shutdown 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 -- 172.21.15.102:0/789918764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 msgr2=0x558be3d00170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3d00170 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff08c01ec30 tx=0x7ff08c010f50 comp rx=0 tx=0).stop 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 -- 172.21.15.102:0/789918764 shutdown_connections 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x558be3d01e60 0x558be3d04250 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558be3c87d80 0x558be3d00170 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558be3c7fb60 0x558be3c848a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 --2- 172.21.15.102:0/789918764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be3c7f240 0x558be3c84360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff07dffb640 0 client.16332 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 -- 172.21.15.102:0/789918764 >> 172.21.15.102:0/789918764 conn(0x558be3c7b560 msgr2=0x558be3c7a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 -- 172.21.15.102:0/789918764 shutdown_connections 2024-04-24T23:25:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 1 -- 172.21.15.102:0/789918764 wait complete. 2024-04-24T23:25:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.997+0000 7ff09f5b1740 20 client.16332 trim_cache size 0 max 16384 2024-04-24T23:25:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:25:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:25:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:25:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:25:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 -- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0800a2da0 msgr2=0x7ff0800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0800a2da0 0x7ff0800a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009fd0 tx=0x7ff08c009ad0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 -- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094100640 msgr2=0x7ff094196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094100640 0x7ff094196e40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff090005ab0 tx=0x7ff090003cd0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 -- 172.21.15.102:0/2170102952 shutdown_connections 2024-04-24T23:25:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff094106650 0x7ff09419b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff094101000 0x7ff094197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff0800a2da0 0x7ff0800a5260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 --2- 172.21.15.102:0/2170102952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff094100640 0x7ff094196e40 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 -- 172.21.15.102:0/2170102952 >> 172.21.15.102:0/2170102952 conn(0x7ff094077760 msgr2=0x7ff094103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 -- 172.21.15.102:0/2170102952 shutdown_connections 2024-04-24T23:25:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:23.998+0000 7ff09c7c8640 1 -- 172.21.15.102:0/2170102952 wait complete. 2024-04-24T23:25:25.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:25 smithi183 ceph-mon[33137]: pgmap v386: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 5.04k op/s 2024-04-24T23:25:25.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:25 smithi102 ceph-mon[29216]: pgmap v386: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 5.04k op/s 2024-04-24T23:25:25.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:25 smithi161 ceph-mon[32709]: pgmap v386: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 5.04k op/s 2024-04-24T23:25:27.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:27 smithi183 ceph-mon[33137]: pgmap v387: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 20 MiB/s wr, 4.96k op/s 2024-04-24T23:25:27.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:27 smithi102 ceph-mon[29216]: pgmap v387: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 20 MiB/s wr, 4.96k op/s 2024-04-24T23:25:27.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:27 smithi161 ceph-mon[32709]: pgmap v387: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 20 MiB/s wr, 4.96k op/s 2024-04-24T23:25:29.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:29 smithi183 ceph-mon[33137]: pgmap v388: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2024-04-24T23:25:29.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:29 smithi102 ceph-mon[29216]: pgmap v388: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2024-04-24T23:25:29.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:29 smithi161 ceph-mon[32709]: pgmap v388: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2024-04-24T23:25:31.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:31 smithi183 ceph-mon[33137]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 24 MiB/s wr, 6.14k op/s 2024-04-24T23:25:31.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:31 smithi102 ceph-mon[29216]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 24 MiB/s wr, 6.14k op/s 2024-04-24T23:25:31.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:31 smithi161 ceph-mon[32709]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 24 MiB/s wr, 6.14k op/s 2024-04-24T23:25:33.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:33 smithi183 ceph-mon[33137]: pgmap v390: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 19 MiB/s wr, 4.91k op/s 2024-04-24T23:25:33.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:33 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:33.580 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:33 smithi102 ceph-mon[29216]: pgmap v390: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 19 MiB/s wr, 4.91k op/s 2024-04-24T23:25:33.580 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:33 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:33.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:33 smithi161 ceph-mon[32709]: pgmap v390: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 19 MiB/s wr, 4.91k op/s 2024-04-24T23:25:33.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:33 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:35.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:35 smithi102 ceph-mon[29216]: pgmap v391: 97 pgs: 97 active+clean; 18 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 25 MiB/s wr, 6.26k op/s 2024-04-24T23:25:35.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:35 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:35 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:35 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:35 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:35 smithi161 ceph-mon[32709]: pgmap v391: 97 pgs: 97 active+clean; 18 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 25 MiB/s wr, 6.26k op/s 2024-04-24T23:25:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:35 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:35 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:35 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:35 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:35 smithi183 ceph-mon[33137]: pgmap v391: 97 pgs: 97 active+clean; 18 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 25 MiB/s wr, 6.26k op/s 2024-04-24T23:25:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:35 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:35 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:35 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:35.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:35 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:36.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:36 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:36.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:36 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:36.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:36 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:36.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:36 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:36 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:36.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:36 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:36.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:36 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:36.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:36 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:36.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:36 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:25:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.171+0000 7ff1f359e640 1 -- 172.21.15.102:0/3067527538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f410e410 msgr2=0x7ff1f411cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.171+0000 7ff1f359e640 1 --2- 172.21.15.102:0/3067527538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f410e410 0x7ff1f411cf30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009be0 tx=0x7ff1ec02a790 comp rx=0 tx=0).stop 2024-04-24T23:25:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.176+0000 7ff1f359e640 1 -- 172.21.15.102:0/3067527538 shutdown_connections 2024-04-24T23:25:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.176+0000 7ff1f359e640 1 --2- 172.21.15.102:0/3067527538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f410e410 0x7ff1f411cf30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.176+0000 7ff1f359e640 1 --2- 172.21.15.102:0/3067527538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f41101e0 0x7ff1f410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.176+0000 7ff1f359e640 1 --2- 172.21.15.102:0/3067527538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1f410fe00 0x7ff1f410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.176+0000 7ff1f359e640 1 -- 172.21.15.102:0/3067527538 >> 172.21.15.102:0/3067527538 conn(0x7ff1f406c680 msgr2=0x7ff1f406ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.180+0000 7ff1f359e640 1 -- 172.21.15.102:0/3067527538 shutdown_connections 2024-04-24T23:25:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.182+0000 7ff1f359e640 1 -- 172.21.15.102:0/3067527538 wait complete. 2024-04-24T23:25:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.183+0000 7ff1f359e640 1 Processor -- start 2024-04-24T23:25:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.183+0000 7ff1f359e640 1 -- start start 2024-04-24T23:25:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.183+0000 7ff1f359e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f410e410 0x7ff1f411c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.183+0000 7ff1f359e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1f410fe00 0x7ff1f41156e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.183+0000 7ff1f359e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f41101e0 0x7ff1f4115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.184+0000 7ff1f359e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41206c0 con 0x7ff1f410fe00 2024-04-24T23:25:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.184+0000 7ff1f359e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4120540 con 0x7ff1f41101e0 2024-04-24T23:25:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.184+0000 7ff1f359e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4120840 con 0x7ff1f410e410 2024-04-24T23:25:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.188+0000 7ff1f2d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f41101e0 0x7ff1f4115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.189+0000 7ff1f2d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f41101e0 0x7ff1f4115c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:48862/0 (socket says 172.21.15.102:48862) 2024-04-24T23:25:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.189+0000 7ff1f2d9d640 1 -- 172.21.15.102:0/2917457903 learned_addr learned my addr 172.21.15.102:0/2917457903 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.190+0000 7ff1f2d9d640 1 -- 172.21.15.102:0/2917457903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f410e410 msgr2=0x7ff1f411c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.190+0000 7ff1f2d9d640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f410e410 0x7ff1f411c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.190+0000 7ff1f2d9d640 1 -- 172.21.15.102:0/2917457903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1f410fe00 msgr2=0x7ff1f41156e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.190+0000 7ff1f2d9d640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1f410fe00 0x7ff1f41156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.191+0000 7ff1f2d9d640 1 -- 172.21.15.102:0/2917457903 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f41162c0 con 0x7ff1f41101e0 2024-04-24T23:25:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.191+0000 7ff1f2d9d640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f41101e0 0x7ff1f4115c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec034040 tx=0x7ff1ec030910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.191+0000 7ff1e37fe640 1 -- 172.21.15.102:0/2917457903 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec041020 con 0x7ff1f41101e0 2024-04-24T23:25:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.193+0000 7ff1e37fe640 1 -- 172.21.15.102:0/2917457903 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1ec030c00 con 0x7ff1f41101e0 2024-04-24T23:25:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.193+0000 7ff1e37fe640 1 -- 172.21.15.102:0/2917457903 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec036040 con 0x7ff1f41101e0 2024-04-24T23:25:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.193+0000 7ff1f359e640 1 -- 172.21.15.102:0/2917457903 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f411a180 con 0x7ff1f41101e0 2024-04-24T23:25:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.193+0000 7ff1f359e640 1 -- 172.21.15.102:0/2917457903 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f411a6c0 con 0x7ff1f41101e0 2024-04-24T23:25:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.199+0000 7ff1e37fe640 1 -- 172.21.15.102:0/2917457903 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7ff1ec044020 con 0x7ff1f41101e0 2024-04-24T23:25:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.200+0000 7ff1fbc47740 1 Processor -- start 2024-04-24T23:25:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.201+0000 7ff1fbc47740 1 -- start start 2024-04-24T23:25:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.201+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef16800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.201+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef20ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.201+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef21410 0x55984ef23800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.201+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55984ee7acb0 con 0x55984ef16d40 2024-04-24T23:25:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.201+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55984ee235b0 con 0x55984ef21410 2024-04-24T23:25:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.201+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55984ee76c70 con 0x55984ef16420 2024-04-24T23:25:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.201+0000 7ff1e37fe640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff1dc0a2e40 0x7ff1dc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.202+0000 7ff1e37fe640 1 -- 172.21.15.102:0/2917457903 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7ff1ec0ef9b0 con 0x7ff1f41101e0 2024-04-24T23:25:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.202+0000 7ff1f2d9d640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff1dc0a2e40 0x7ff1dc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.203+0000 7ff1f2d9d640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff1dc0a2e40 0x7ff1dc0a5300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0308b0 tx=0x7ff1ec009500 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.209+0000 7ff1f1d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef16800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.209+0000 7ff1f1d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef16800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:49498/0 (socket says 172.21.15.102:49498) 2024-04-24T23:25:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.209+0000 7ff1f259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef20ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.209+0000 7ff1f1d9b640 1 -- 172.21.15.102:0/2319061249 learned_addr learned my addr 172.21.15.102:0/2319061249 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.209+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/2319061249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef21410 0x55984ef23800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.210+0000 7ff1f1d9b640 1 -- 172.21.15.102:0/2319061249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef21410 msgr2=0x55984ef23800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.210+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/2319061249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef21410 0x55984ef23800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.210+0000 7ff1f1d9b640 1 -- 172.21.15.102:0/2319061249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 msgr2=0x55984ef20ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.210+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/2319061249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef20ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.210+0000 7ff1f1d9b640 1 -- 172.21.15.102:0/2319061249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984eb8f210 con 0x55984ef16420 2024-04-24T23:25:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.211+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/2319061249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef16800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80137d0 tx=0x7ff1e8009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=721629f951e48be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.217+0000 7ff1e0ff9640 1 -- 172.21.15.102:0/2319061249 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80132d0 con 0x55984ef16420 2024-04-24T23:25:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.217+0000 7ff1e0ff9640 1 -- 172.21.15.102:0/2319061249 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1e8013e10 con 0x55984ef16420 2024-04-24T23:25:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.218+0000 7ff1fbc47740 1 -- 172.21.15.102:0/2319061249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 msgr2=0x55984ef16800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.218+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/2319061249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef16800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80137d0 tx=0x7ff1e8009620 comp rx=0 tx=0).stop 2024-04-24T23:25:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.223+0000 7ff1fbc47740 1 -- 172.21.15.102:0/2319061249 shutdown_connections 2024-04-24T23:25:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.223+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/2319061249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef21410 0x55984ef23800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.223+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/2319061249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef20ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.223+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/2319061249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef16800 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.223+0000 7ff1fbc47740 1 -- 172.21.15.102:0/2319061249 >> 172.21.15.102:0/2319061249 conn(0x55984ef12740 msgr2=0x55984ef12b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.230+0000 7ff1fbc47740 1 -- 172.21.15.102:0/2319061249 shutdown_connections 2024-04-24T23:25:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.235+0000 7ff1fbc47740 1 -- 172.21.15.102:0/2319061249 wait complete. 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.236+0000 7ff1fbc47740 1 -- start start 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.236+0000 7ff1fbc47740 1 Processor -- start 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.236+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef20ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.236+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef1e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.236+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef21410 0x55984ef1e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.236+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55984ef241f0 con 0x55984ef16d40 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.236+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55984ee36b50 con 0x55984ef21410 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.236+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55984ef24370 con 0x55984ef16420 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.236+0000 7ff1fbc47740 10 client.0 _ll_register_callbacks cb 0x55984ee71ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.239+0000 7ff1f259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef1e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.239+0000 7ff1f259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef1e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49360/0 (socket says 172.21.15.102:49360) 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.239+0000 7ff1f259c640 1 -- 172.21.15.102:0/1470184400 learned_addr learned my addr 172.21.15.102:0/1470184400 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.239+0000 7ff1f259c640 1 -- 172.21.15.102:0/1470184400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 msgr2=0x55984ef20ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.239+0000 7ff1f259c640 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef20ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.239+0000 7ff1f259c640 1 -- 172.21.15.102:0/1470184400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef21410 msgr2=0x55984ef1e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.239+0000 7ff1f259c640 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef21410 0x55984ef1e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.239+0000 7ff1f259c640 1 -- 172.21.15.102:0/1470184400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984ec5b620 con 0x55984ef16d40 2024-04-24T23:25:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.240+0000 7ff1d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:25:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.240+0000 7ff1f259c640 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef1e260 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e240 tx=0x7ff1e400e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166c196c4c423784 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.241+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/1470184400 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4017070 con 0x55984ef16d40 2024-04-24T23:25:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.241+0000 7ff1fbc47740 1 -- 172.21.15.102:0/1470184400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e8013e10 con 0x55984ef16d40 2024-04-24T23:25:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.241+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/1470184400 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7ff1e400f040 con 0x55984ef16d40 2024-04-24T23:25:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.241+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/1470184400 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff1e4021050 con 0x55984ef16d40 2024-04-24T23:25:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.241+0000 7ff1fbc47740 10 client.16338 fetch_fsmap learned FSMap version 26 2024-04-24T23:25:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.242+0000 7ff1fbc47740 1 -- 172.21.15.102:0/1470184400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55984ef30460 con 0x55984ef16d40 2024-04-24T23:25:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.242+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/1470184400 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1e8013e10 con 0x55984ef16d40 2024-04-24T23:25:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.242+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/1470184400 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff1e401c670 con 0x55984ef16d40 2024-04-24T23:25:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.242+0000 7ff1fbc47740 10 client.16338 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:25:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.242+0000 7ff1fbc47740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:25:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.242+0000 7ff1fbc47740 20 client.16338 populate_metadata read hostname 'smithi102' 2024-04-24T23:25:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.243+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55984ef31d60 0x55984ef34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.243+0000 7ff1fbc47740 4 client.16338 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.243+0000 7ff1fbc47740 1 -- 172.21.15.102:0/1470184400 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55984ee881f0 con 0x55984ef31d60 2024-04-24T23:25:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.244+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55984ef31d60 0x55984ef34220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.246+0000 7ff1d9ffb640 10 client.16338 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.246+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55984ef31d60 0x55984ef34220 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72573bd5b0d749be server_cookie=8a070d4981da56d in_seq=0 out_seq=0 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.247+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/1470184400 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55984ee881f0 con 0x55984ef31d60 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.247+0000 7ff1d9ffb640 10 client.16338 handle_command_reply: tid=0 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.247+0000 7ff1fbc47740 1 client.16338 shutdown 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.250+0000 7ff1fbc47740 1 -- 172.21.15.102:0/1470184400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 msgr2=0x55984ef1e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.250+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef1e260 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e240 tx=0x7ff1e400e700 comp rx=0 tx=0).stop 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.250+0000 7ff1fbc47740 1 -- 172.21.15.102:0/1470184400 shutdown_connections 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.250+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef21410 0x55984ef1e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.250+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55984ef31d60 0x55984ef34220 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.250+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef16d40 0x55984ef1e260 secure :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e240 tx=0x7ff1e400e700 comp rx=0 tx=0).stop 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.250+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/1470184400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef20ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.250+0000 7ff1fbc47740 1 -- 172.21.15.102:0/1470184400 >> 172.21.15.102:0/1470184400 conn(0x55984ef12740 msgr2=0x55984ef18420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.251+0000 7ff1d9ffb640 0 client.16338 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.251+0000 7ff1fbc47740 1 -- 172.21.15.102:0/1470184400 shutdown_connections 2024-04-24T23:25:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.253+0000 7ff1fbc47740 1 -- 172.21.15.102:0/1470184400 wait complete. 2024-04-24T23:25:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.253+0000 7ff1fbc47740 20 client.16338 trim_cache size 0 max 16384 2024-04-24T23:25:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.299+0000 7ff1fbc47740 1 Processor -- start 2024-04-24T23:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.307+0000 7ff1fbc47740 1 -- start start 2024-04-24T23:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.307+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef16420 0x55984ef2d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.307+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef21410 0x55984ef20990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.308+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef20ed0 0x55984ef39b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.308+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4011450 con 0x55984ef20ed0 2024-04-24T23:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.308+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d400ed20 con 0x55984ef16420 2024-04-24T23:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.308+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d400cb10 con 0x55984ef21410 2024-04-24T23:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.310+0000 7ff1f1d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef16420 0x55984ef2d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.310+0000 7ff1f1d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef16420 0x55984ef2d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:48898/0 (socket says 172.21.15.102:48898) 2024-04-24T23:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.310+0000 7ff1f1d9b640 1 -- 172.21.15.102:0/3036601894 learned_addr learned my addr 172.21.15.102:0/3036601894 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.310+0000 7ff1f259c640 1 --2- 172.21.15.102:0/3036601894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef21410 0x55984ef20990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.312+0000 7ff1f259c640 1 -- 172.21.15.102:0/3036601894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef16420 msgr2=0x55984ef2d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.312+0000 7ff1f259c640 1 --2- 172.21.15.102:0/3036601894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef16420 0x55984ef2d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.312+0000 7ff1f259c640 1 -- 172.21.15.102:0/3036601894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef20ed0 msgr2=0x55984ef39b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.312+0000 7ff1f259c640 1 --2- 172.21.15.102:0/3036601894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef20ed0 0x55984ef39b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.312+0000 7ff1f259c640 1 -- 172.21.15.102:0/3036601894 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984ef1ed10 con 0x55984ef21410 2024-04-24T23:25:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.316+0000 7ff1f259c640 1 --2- 172.21.15.102:0/3036601894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef21410 0x55984ef20990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4010e80 tx=0x7ff1e4001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b91881b70b7c861 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.317+0000 7ff1dbfff640 1 -- 172.21.15.102:0/3036601894 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4021050 con 0x55984ef21410 2024-04-24T23:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.317+0000 7ff1dbfff640 1 -- 172.21.15.102:0/3036601894 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1e400e0b0 con 0x55984ef21410 2024-04-24T23:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.317+0000 7ff1dbfff640 1 -- 172.21.15.102:0/3036601894 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4017070 con 0x55984ef21410 2024-04-24T23:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.319+0000 7ff1fbc47740 1 -- 172.21.15.102:0/3036601894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef21410 msgr2=0x55984ef20990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.319+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/3036601894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef21410 0x55984ef20990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4010e80 tx=0x7ff1e4001e50 comp rx=0 tx=0).stop 2024-04-24T23:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.321+0000 7ff1fbc47740 1 -- 172.21.15.102:0/3036601894 shutdown_connections 2024-04-24T23:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.321+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/3036601894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef20ed0 0x55984ef39b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.321+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/3036601894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef21410 0x55984ef20990 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4010e80 tx=0x7ff1e4001e50 comp rx=0 tx=0).stop 2024-04-24T23:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.321+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/3036601894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef16420 0x55984ef2d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.321+0000 7ff1fbc47740 1 -- 172.21.15.102:0/3036601894 >> 172.21.15.102:0/3036601894 conn(0x55984ef12740 msgr2=0x55984ef12b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.327+0000 7ff1fbc47740 1 -- 172.21.15.102:0/3036601894 shutdown_connections 2024-04-24T23:25:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.328+0000 7ff1fbc47740 1 -- 172.21.15.102:0/3036601894 wait complete. 2024-04-24T23:25:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.330+0000 7ff1fbc47740 1 -- start start 2024-04-24T23:25:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.330+0000 7ff1fbc47740 1 Processor -- start 2024-04-24T23:25:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.331+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef1aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.331+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef20ed0 0x55984ef1b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.331+0000 7ff1fbc47740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef1ba70 0x55984ef973a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.331+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d400d1e0 con 0x55984ef1ba70 2024-04-24T23:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.331+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d400e650 con 0x55984ef20ed0 2024-04-24T23:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.331+0000 7ff1f1d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef1aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.331+0000 7ff1fbc47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d400d8b0 con 0x55984ef16420 2024-04-24T23:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.331+0000 7ff1f1d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef1aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:49536/0 (socket says 172.21.15.102:49536) 2024-04-24T23:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.331+0000 7ff1f1d9b640 1 -- 172.21.15.102:0/260717918 learned_addr learned my addr 172.21.15.102:0/260717918 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.332+0000 7ff1fbc47740 10 client.0 _ll_register_callbacks cb 0x7ff1e4009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.333+0000 7ff1f259c640 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef20ed0 0x55984ef1b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.336+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef1ba70 0x55984ef973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.337+0000 7ff1f1d9b640 1 -- 172.21.15.102:0/260717918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef20ed0 msgr2=0x55984ef1b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.337+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef20ed0 0x55984ef1b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.337+0000 7ff1f1d9b640 1 -- 172.21.15.102:0/260717918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef1ba70 msgr2=0x55984ef973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.337+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef1ba70 0x55984ef973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.337+0000 7ff1f1d9b640 1 -- 172.21.15.102:0/260717918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984ef12570 con 0x55984ef16420 2024-04-24T23:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.337+0000 7ff1f1d9b640 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef1aff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80137d0 tx=0x7ff1e8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7bd5c6fc30d7629 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.340+0000 7ff1d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.342+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/260717918 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8027dd0 con 0x55984ef16420 2024-04-24T23:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.342+0000 7ff1fbc47740 1 -- 172.21.15.102:0/260717918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55984ee82540 con 0x55984ef16420 2024-04-24T23:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.342+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/260717918 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7ff1e800e720 con 0x55984ef16420 2024-04-24T23:25:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.350+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/260717918 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff1e8037050 con 0x55984ef16420 2024-04-24T23:25:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.350+0000 7ff1fbc47740 10 client.25684 fetch_fsmap learned FSMap version 26 2024-04-24T23:25:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.350+0000 7ff1fbc47740 1 -- 172.21.15.102:0/260717918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55984ef98a00 con 0x55984ef16420 2024-04-24T23:25:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.352+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/260717918 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55984ee82540 con 0x55984ef16420 2024-04-24T23:25:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.352+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/260717918 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff1e80053a0 con 0x55984ef16420 2024-04-24T23:25:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.353+0000 7ff1fbc47740 10 client.25684 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:25:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.353+0000 7ff1fbc47740 10 client.25684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:25:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.353+0000 7ff1fbc47740 20 client.25684 populate_metadata read hostname 'smithi102' 2024-04-24T23:25:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.353+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55984ef990a0 0x55984ef9b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.353+0000 7ff1fbc47740 4 client.25684 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:25:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.353+0000 7ff1fbc47740 1 -- 172.21.15.102:0/260717918 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55984ee008e0 con 0x55984ef990a0 2024-04-24T23:25:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.360+0000 7ff1f259c640 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55984ef990a0 0x55984ef9b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.366+0000 7ff1f259c640 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55984ef990a0 0x55984ef9b490 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ae1b17be34e801 server_cookie=aa96762f4dfaf2c1 in_seq=0 out_seq=0 2024-04-24T23:25:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.367+0000 7ff1d9ffb640 10 client.25684 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.545+0000 7ff1d9ffb640 1 -- 172.21.15.102:0/260717918 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff1e402a040 con 0x55984ef990a0 2024-04-24T23:25:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.545+0000 7ff1d9ffb640 10 client.25684 handle_command_reply: tid=0 2024-04-24T23:25:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.545+0000 7ff1fbc47740 1 client.25684 shutdown 2024-04-24T23:25:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.545+0000 7ff1fbc47740 1 -- 172.21.15.102:0/260717918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 msgr2=0x55984ef1aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.545+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef1aff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80137d0 tx=0x7ff1e8009260 comp rx=0 tx=0).stop 2024-04-24T23:25:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.547+0000 7ff1fbc47740 1 -- 172.21.15.102:0/260717918 shutdown_connections 2024-04-24T23:25:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.547+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55984ef990a0 0x55984ef9b490 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.547+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55984ef1ba70 0x55984ef973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.547+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55984ef20ed0 0x55984ef1b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.547+0000 7ff1fbc47740 1 --2- 172.21.15.102:0/260717918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55984ef16420 0x55984ef1aff0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.547+0000 7ff1fbc47740 1 -- 172.21.15.102:0/260717918 >> 172.21.15.102:0/260717918 conn(0x55984ef12740 msgr2=0x55984ef1caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.547+0000 7ff1d9ffb640 0 client.25684 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.549+0000 7ff1fbc47740 1 -- 172.21.15.102:0/260717918 shutdown_connections 2024-04-24T23:25:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.549+0000 7ff1fbc47740 1 -- 172.21.15.102:0/260717918 wait complete. 2024-04-24T23:25:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.549+0000 7ff1fbc47740 20 client.25684 trim_cache size 0 max 16384 2024-04-24T23:25:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:25:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:25:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:25:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:25:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.550+0000 7ff1f359e640 1 -- 172.21.15.102:0/2917457903 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff1dc0a2e40 msgr2=0x7ff1dc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.550+0000 7ff1f359e640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff1dc0a2e40 0x7ff1dc0a5300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0308b0 tx=0x7ff1ec009500 comp rx=0 tx=0).stop 2024-04-24T23:25:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.550+0000 7ff1f359e640 1 -- 172.21.15.102:0/2917457903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f41101e0 msgr2=0x7ff1f4115c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.550+0000 7ff1f359e640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f41101e0 0x7ff1f4115c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec034040 tx=0x7ff1ec030910 comp rx=0 tx=0).stop 2024-04-24T23:25:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.552+0000 7ff1f359e640 1 -- 172.21.15.102:0/2917457903 shutdown_connections 2024-04-24T23:25:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.552+0000 7ff1f359e640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1f41101e0 0x7ff1f4115c20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.552+0000 7ff1f359e640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7ff1dc0a2e40 0x7ff1dc0a5300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.552+0000 7ff1f359e640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1f410fe00 0x7ff1f41156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.552+0000 7ff1f359e640 1 --2- 172.21.15.102:0/2917457903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f410e410 0x7ff1f411c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.552+0000 7ff1f359e640 1 -- 172.21.15.102:0/2917457903 >> 172.21.15.102:0/2917457903 conn(0x7ff1f406c680 msgr2=0x7ff1f4070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.557+0000 7ff1f359e640 1 -- 172.21.15.102:0/2917457903 shutdown_connections 2024-04-24T23:25:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:37.560+0000 7ff1f359e640 1 -- 172.21.15.102:0/2917457903 wait complete. 2024-04-24T23:25:37.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:37 smithi161 ceph-mon[32709]: pgmap v392: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:25:37.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:37 smithi183 ceph-mon[33137]: pgmap v392: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:25:37.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:37 smithi102 ceph-mon[29216]: pgmap v392: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 22 MiB/s wr, 5.50k op/s 2024-04-24T23:25:39.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:39 smithi161 ceph-mon[32709]: pgmap v393: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 19 MiB/s wr, 4.73k op/s 2024-04-24T23:25:39.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:39 smithi183 ceph-mon[33137]: pgmap v393: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 19 MiB/s wr, 4.73k op/s 2024-04-24T23:25:39.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:39 smithi102 ceph-mon[29216]: pgmap v393: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 19 MiB/s wr, 4.73k op/s 2024-04-24T23:25:41.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:41 smithi161 ceph-mon[32709]: pgmap v394: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 27 MiB/s wr, 6.73k op/s 2024-04-24T23:25:41.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:41 smithi183 ceph-mon[33137]: pgmap v394: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 27 MiB/s wr, 6.73k op/s 2024-04-24T23:25:41.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:41 smithi102 ceph-mon[29216]: pgmap v394: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 27 MiB/s wr, 6.73k op/s 2024-04-24T23:25:43.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:43 smithi161 ceph-mon[32709]: pgmap v395: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2024-04-24T23:25:43.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:43 smithi183 ceph-mon[33137]: pgmap v395: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2024-04-24T23:25:43.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:43 smithi102 ceph-mon[29216]: pgmap v395: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2024-04-24T23:25:45.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:45 smithi161 ceph-mon[32709]: pgmap v396: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 25 MiB/s wr, 6.24k op/s 2024-04-24T23:25:45.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:45 smithi183 ceph-mon[33137]: pgmap v396: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 25 MiB/s wr, 6.24k op/s 2024-04-24T23:25:45.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:45 smithi102 ceph-mon[29216]: pgmap v396: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 25 MiB/s wr, 6.24k op/s 2024-04-24T23:25:47.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:47 smithi161 ceph-mon[32709]: pgmap v397: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 21 MiB/s wr, 5.32k op/s 2024-04-24T23:25:47.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:47 smithi183 ceph-mon[33137]: pgmap v397: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 21 MiB/s wr, 5.32k op/s 2024-04-24T23:25:47.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:47 smithi102 ceph-mon[29216]: pgmap v397: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 21 MiB/s wr, 5.32k op/s 2024-04-24T23:25:49.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:49 smithi161 ceph-mon[32709]: pgmap v398: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2024-04-24T23:25:49.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:49 smithi183 ceph-mon[33137]: pgmap v398: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2024-04-24T23:25:49.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:49 smithi102 ceph-mon[29216]: pgmap v398: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2024-04-24T23:25:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.742+0000 7f821effd640 1 -- 172.21.15.102:0/2856391132 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210037070 con 0x7f82201089b0 2024-04-24T23:25:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.743+0000 7f8225626640 1 -- 172.21.15.102:0/2856391132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201089b0 msgr2=0x7f8220108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.743+0000 7f8225626640 1 --2- 172.21.15.102:0/2856391132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201089b0 0x7f8220108d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8210009860 tx=0x7f821002a360 comp rx=0 tx=0).stop 2024-04-24T23:25:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.744+0000 7f8225626640 1 -- 172.21.15.102:0/2856391132 shutdown_connections 2024-04-24T23:25:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.744+0000 7f8225626640 1 --2- 172.21.15.102:0/2856391132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f822010d2c0 0x7f8220114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.744+0000 7f8225626640 1 --2- 172.21.15.102:0/2856391132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8220109360 0x7f822010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.744+0000 7f8225626640 1 --2- 172.21.15.102:0/2856391132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201089b0 0x7f8220108d90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.744+0000 7f8225626640 1 -- 172.21.15.102:0/2856391132 >> 172.21.15.102:0/2856391132 conn(0x7f822006c680 msgr2=0x7f822006ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.750+0000 7f8225626640 1 -- 172.21.15.102:0/2856391132 shutdown_connections 2024-04-24T23:25:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.753+0000 7f8225626640 1 -- 172.21.15.102:0/2856391132 wait complete. 2024-04-24T23:25:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.754+0000 7f8225626640 1 Processor -- start 2024-04-24T23:25:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.755+0000 7f8225626640 1 -- start start 2024-04-24T23:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.755+0000 7f8225626640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201089b0 0x7f8220122550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.755+0000 7f8225626640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f822010d2c0 0x7f8220122a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.755+0000 7f8225626640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82201140f0 0x7f8220114580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.755+0000 7f8225626640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8220117390 con 0x7f82201140f0 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.755+0000 7f8225626640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8220117210 con 0x7f82201089b0 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.755+0000 7f8225626640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8220117510 con 0x7f822010d2c0 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f8224e25640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82201140f0 0x7f8220114580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f8224e25640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82201140f0 0x7f8220114580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44080/0 (socket says 172.21.15.102:44080) 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f8224e25640 1 -- 172.21.15.102:0/2787943039 learned_addr learned my addr 172.21.15.102:0/2787943039 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f821f7fe640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f822010d2c0 0x7f8220122a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f821ffff640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201089b0 0x7f8220122550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f821ffff640 1 -- 172.21.15.102:0/2787943039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f822010d2c0 msgr2=0x7f8220122a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f821ffff640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f822010d2c0 0x7f8220122a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f821ffff640 1 -- 172.21.15.102:0/2787943039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82201140f0 msgr2=0x7f8220114580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f821ffff640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82201140f0 0x7f8220114580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f821ffff640 1 -- 172.21.15.102:0/2787943039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8220114d80 con 0x7f82201089b0 2024-04-24T23:25:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f821ffff640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201089b0 0x7f8220122550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8210000c00 tx=0x7f8210005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f821d7fa640 1 -- 172.21.15.102:0/2787943039 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210040020 con 0x7f82201089b0 2024-04-24T23:25:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.756+0000 7f8225626640 1 -- 172.21.15.102:0/2787943039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822011dff0 con 0x7f82201089b0 2024-04-24T23:25:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.757+0000 7f821d7fa640 1 -- 172.21.15.102:0/2787943039 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8210004b70 con 0x7f82201089b0 2024-04-24T23:25:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.757+0000 7f821d7fa640 1 -- 172.21.15.102:0/2787943039 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210037040 con 0x7f82201089b0 2024-04-24T23:25:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.757+0000 7f8225626640 1 -- 172.21.15.102:0/2787943039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822011e530 con 0x7f82201089b0 2024-04-24T23:25:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8229592740 1 Processor -- start 2024-04-24T23:25:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8229592740 1 -- start start 2024-04-24T23:25:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8229592740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f0c2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8229592740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 0x56133f0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8229592740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8229592740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56133f026cb0 con 0x56133f0cd410 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8229592740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56133efcf5b0 con 0x56133f0c2d40 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8229592740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56133f022c70 con 0x56133f0c2420 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f821d7fa640 1 -- 172.21.15.102:0/2787943039 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8210043020 con 0x7f82201089b0 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8224e25640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f0c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f821f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8224e25640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f0c2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:54588/0 (socket says 172.21.15.102:54588) 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f8224e25640 1 -- 172.21.15.102:0/1554804607 learned_addr learned my addr 172.21.15.102:0/1554804607 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.761+0000 7f821ffff640 1 --2- 172.21.15.102:0/1554804607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 0x56133f0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.762+0000 7f821f7fe640 1 -- 172.21.15.102:0/1554804607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 msgr2=0x56133f0c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.762+0000 7f821f7fe640 1 --2- 172.21.15.102:0/1554804607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f0c2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.762+0000 7f821f7fe640 1 -- 172.21.15.102:0/1554804607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 msgr2=0x56133f0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.762+0000 7f821f7fe640 1 --2- 172.21.15.102:0/1554804607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 0x56133f0cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.762+0000 7f821f7fe640 1 -- 172.21.15.102:0/1554804607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133ed3b210 con 0x56133f0cd410 2024-04-24T23:25:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.762+0000 7f821d7fa640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82080a2d70 0x7f82080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.763+0000 7f821d7fa640 1 -- 172.21.15.102:0/2787943039 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7f82100ef460 con 0x7f82201089b0 2024-04-24T23:25:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.763+0000 7f821f7fe640 1 --2- 172.21.15.102:0/1554804607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f0cf800 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f822011a8b0 tx=0x7f821400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdc556af769d48e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.763+0000 7f82067fc640 1 -- 172.21.15.102:0/1554804607 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821400be30 con 0x56133f0cd410 2024-04-24T23:25:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.763+0000 7f8224e25640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82080a2d70 0x7f82080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.764+0000 7f8224e25640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82080a2d70 0x7f82080a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82180133b0 tx=0x7f821800c040 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.764+0000 7f82067fc640 1 -- 172.21.15.102:0/1554804607 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8214019070 con 0x56133f0cd410 2024-04-24T23:25:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.764+0000 7f82067fc640 1 -- 172.21.15.102:0/1554804607 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214014d00 con 0x56133f0cd410 2024-04-24T23:25:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.765+0000 7f8229592740 1 -- 172.21.15.102:0/1554804607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 msgr2=0x56133f0cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.765+0000 7f8229592740 1 --2- 172.21.15.102:0/1554804607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f0cf800 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f822011a8b0 tx=0x7f821400bbe0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.766+0000 7f8229592740 1 -- 172.21.15.102:0/1554804607 shutdown_connections 2024-04-24T23:25:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.766+0000 7f8229592740 1 --2- 172.21.15.102:0/1554804607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f0cf800 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.766+0000 7f8229592740 1 --2- 172.21.15.102:0/1554804607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 0x56133f0cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.766+0000 7f8229592740 1 --2- 172.21.15.102:0/1554804607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f0c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.766+0000 7f8229592740 1 -- 172.21.15.102:0/1554804607 >> 172.21.15.102:0/1554804607 conn(0x56133f0be740 msgr2=0x56133f0beb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.766+0000 7f8229592740 1 -- 172.21.15.102:0/1554804607 shutdown_connections 2024-04-24T23:25:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.766+0000 7f8229592740 1 -- 172.21.15.102:0/1554804607 wait complete. 2024-04-24T23:25:50.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.766+0000 7f8229592740 1 -- start start 2024-04-24T23:25:50.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.767+0000 7f8229592740 1 Processor -- start 2024-04-24T23:25:50.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.767+0000 7f8229592740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f0ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.769 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.768+0000 7f8229592740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2d40 0x56133f0ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.768+0000 7f8229592740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0cd410 0x56133f0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.768+0000 7f8229592740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56133f0d0200 con 0x56133f0c2d40 2024-04-24T23:25:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.768+0000 7f8229592740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56133efe2b50 con 0x56133f0cd410 2024-04-24T23:25:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.768+0000 7f8229592740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56133f0d0380 con 0x56133f0c2420 2024-04-24T23:25:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.768+0000 7f8229592740 10 client.0 _ll_register_callbacks cb 0x56133f01dca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.768+0000 7f821ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f0ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.769+0000 7f821f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2d40 0x56133f0ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.769+0000 7f821f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2d40 0x56133f0ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44104/0 (socket says 172.21.15.102:44104) 2024-04-24T23:25:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.769+0000 7f821f7fe640 1 -- 172.21.15.102:0/1049958623 learned_addr learned my addr 172.21.15.102:0/1049958623 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.769+0000 7f8224e25640 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0cd410 0x56133f0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.769+0000 7f821f7fe640 1 -- 172.21.15.102:0/1049958623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 msgr2=0x56133f0ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.769+0000 7f821f7fe640 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f0ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.769+0000 7f821f7fe640 1 -- 172.21.15.102:0/1049958623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0cd410 msgr2=0x56133f0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.769+0000 7f821f7fe640 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0cd410 0x56133f0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.769+0000 7f821f7fe640 1 -- 172.21.15.102:0/1049958623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56133ee07620 con 0x56133f0c2d40 2024-04-24T23:25:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.770+0000 7f821f7fe640 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2d40 0x56133f0ca8c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8214006db0 tx=0x7f8214014660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5f4aa962c1b4cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.770+0000 7f81ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:25:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.770+0000 7f81ff7fe640 1 -- 172.21.15.102:0/1049958623 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214023070 con 0x56133f0c2d40 2024-04-24T23:25:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.770+0000 7f8229592740 1 -- 172.21.15.102:0/1049958623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8214019070 con 0x56133f0c2d40 2024-04-24T23:25:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.771+0000 7f81ff7fe640 1 -- 172.21.15.102:0/1049958623 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7f821400d920 con 0x56133f0c2d40 2024-04-24T23:25:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.772+0000 7f81ff7fe640 1 -- 172.21.15.102:0/1049958623 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f821402f050 con 0x56133f0c2d40 2024-04-24T23:25:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.772+0000 7f8229592740 10 client.16353 fetch_fsmap learned FSMap version 26 2024-04-24T23:25:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.772+0000 7f8229592740 1 -- 172.21.15.102:0/1049958623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56133f0dc420 con 0x56133f0c2d40 2024-04-24T23:25:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.773+0000 7f81ff7fe640 1 -- 172.21.15.102:0/1049958623 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8214019070 con 0x56133f0c2d40 2024-04-24T23:25:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.773+0000 7f81ff7fe640 1 -- 172.21.15.102:0/1049958623 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8214013070 con 0x56133f0c2d40 2024-04-24T23:25:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.773+0000 7f8229592740 10 client.16353 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:25:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.773+0000 7f8229592740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:25:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.773+0000 7f8229592740 20 client.16353 populate_metadata read hostname 'smithi102' 2024-04-24T23:25:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.773+0000 7f8229592740 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56133f0ddda0 0x56133f0e0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.773+0000 7f8229592740 4 client.16353 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:25:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.773+0000 7f8229592740 1 -- 172.21.15.102:0/1049958623 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56133f0341f0 con 0x56133f0ddda0 2024-04-24T23:25:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.773+0000 7f821ffff640 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56133f0ddda0 0x56133f0e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.774+0000 7f821ffff640 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56133f0ddda0 0x56133f0e0220 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9006e7f52ab3898 server_cookie=104f175a2fd02065 in_seq=0 out_seq=0 2024-04-24T23:25:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.775+0000 7f81ff7fe640 10 client.16353 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.778+0000 7f81ff7fe640 1 -- 172.21.15.102:0/1049958623 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56133f0341f0 con 0x56133f0ddda0 2024-04-24T23:25:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.778+0000 7f81ff7fe640 10 client.16353 handle_command_reply: tid=0 2024-04-24T23:25:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.778+0000 7f8229592740 1 client.16353 shutdown 2024-04-24T23:25:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.781+0000 7f8229592740 1 -- 172.21.15.102:0/1049958623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2d40 msgr2=0x56133f0ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.781+0000 7f8229592740 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2d40 0x56133f0ca8c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8214006db0 tx=0x7f8214014660 comp rx=0 tx=0).stop 2024-04-24T23:25:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.782+0000 7f8229592740 1 -- 172.21.15.102:0/1049958623 shutdown_connections 2024-04-24T23:25:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.782+0000 7f8229592740 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0cd410 0x56133f0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.782+0000 7f8229592740 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56133f0ddda0 0x56133f0e0220 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.782+0000 7f8229592740 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2d40 0x56133f0ca8c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.782+0000 7f81ff7fe640 0 client.16353 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.782+0000 7f8229592740 1 --2- 172.21.15.102:0/1049958623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f0ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.782+0000 7f8229592740 1 -- 172.21.15.102:0/1049958623 >> 172.21.15.102:0/1049958623 conn(0x56133f0be740 msgr2=0x56133f0c4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.783+0000 7f8229592740 1 -- 172.21.15.102:0/1049958623 shutdown_connections 2024-04-24T23:25:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.783+0000 7f8229592740 1 -- 172.21.15.102:0/1049958623 wait complete. 2024-04-24T23:25:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.783+0000 7f8229592740 20 client.16353 trim_cache size 0 max 16384 2024-04-24T23:25:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.823+0000 7f8229592740 1 Processor -- start 2024-04-24T23:25:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8229592740 1 -- start start 2024-04-24T23:25:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8229592740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2420 0x56133f0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8229592740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 0x56133f0d90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8229592740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0cd410 0x56133f0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8229592740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82201c7340 con 0x56133f0c2420 2024-04-24T23:25:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8229592740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82201c4c10 con 0x56133f0c2d40 2024-04-24T23:25:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8229592740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82201c2a80 con 0x56133f0cd410 2024-04-24T23:25:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8224e25640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0cd410 0x56133f0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8224e25640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0cd410 0x56133f0c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:54600/0 (socket says 172.21.15.102:54600) 2024-04-24T23:25:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f821ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2420 0x56133f0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f8224e25640 1 -- 172.21.15.102:0/717541235 learned_addr learned my addr 172.21.15.102:0/717541235 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:50.826 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.824+0000 7f821f7fe640 1 --2- 172.21.15.102:0/717541235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 0x56133f0d90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.825+0000 7f821ffff640 1 -- 172.21.15.102:0/717541235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0cd410 msgr2=0x56133f0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.825+0000 7f821ffff640 1 --2- 172.21.15.102:0/717541235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0cd410 0x56133f0c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.825+0000 7f821ffff640 1 -- 172.21.15.102:0/717541235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 msgr2=0x56133f0d90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.825+0000 7f821ffff640 1 --2- 172.21.15.102:0/717541235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 0x56133f0d90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.825+0000 7f821ffff640 1 -- 172.21.15.102:0/717541235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133f0bcc20 con 0x56133f0c2420 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.825+0000 7f821ffff640 1 --2- 172.21.15.102:0/717541235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2420 0x56133f0ccab0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f82100ecc20 tx=0x7f8210033f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1647a28b191d2f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.826+0000 7f8205ffb640 1 -- 172.21.15.102:0/717541235 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82100bb530 con 0x56133f0c2420 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.826+0000 7f8205ffb640 1 -- 172.21.15.102:0/717541235 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f821003c410 con 0x56133f0c2420 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.827+0000 7f8229592740 1 -- 172.21.15.102:0/717541235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2420 msgr2=0x56133f0ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.827+0000 7f8229592740 1 --2- 172.21.15.102:0/717541235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2420 0x56133f0ccab0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f82100ecc20 tx=0x7f8210033f50 comp rx=0 tx=0).stop 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.829+0000 7f8229592740 1 -- 172.21.15.102:0/717541235 shutdown_connections 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.829+0000 7f8229592740 1 --2- 172.21.15.102:0/717541235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0cd410 0x56133f0c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.829+0000 7f8229592740 1 --2- 172.21.15.102:0/717541235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0c2d40 0x56133f0d90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.829+0000 7f8229592740 1 --2- 172.21.15.102:0/717541235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0c2420 0x56133f0ccab0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.829+0000 7f8229592740 1 -- 172.21.15.102:0/717541235 >> 172.21.15.102:0/717541235 conn(0x56133f0be740 msgr2=0x56133f0c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.830+0000 7f8229592740 1 -- 172.21.15.102:0/717541235 shutdown_connections 2024-04-24T23:25:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.830+0000 7f8229592740 1 -- 172.21.15.102:0/717541235 wait complete. 2024-04-24T23:25:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.830+0000 7f8229592740 1 -- start start 2024-04-24T23:25:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.831+0000 7f8229592740 1 Processor -- start 2024-04-24T23:25:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.831+0000 7f8229592740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f13c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.831+0000 7f8229592740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f13cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.831+0000 7f8229592740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0d0730 0x56133f0d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f8229592740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82201c30d0 con 0x56133f0cd410 2024-04-24T23:25:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f8229592740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82201c4540 con 0x56133f0d0730 2024-04-24T23:25:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f8229592740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82201c37a0 con 0x56133f0c2420 2024-04-24T23:25:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f8229592740 10 client.0 _ll_register_callbacks cb 0x7f82100f2de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:25:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f8224e25640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0d0730 0x56133f0d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f8224e25640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0d0730 0x56133f0d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:41184/0 (socket says 172.21.15.102:41184) 2024-04-24T23:25:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f8224e25640 1 -- 172.21.15.102:0/2657570858 learned_addr learned my addr 172.21.15.102:0/2657570858 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:25:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f821ffff640 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f13c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f821f7fe640 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f13cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f821ffff640 1 -- 172.21.15.102:0/2657570858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0d0730 msgr2=0x56133f0d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f821ffff640 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0d0730 0x56133f0d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f821ffff640 1 -- 172.21.15.102:0/2657570858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 msgr2=0x56133f13cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:25:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f821ffff640 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f13cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.832+0000 7f821ffff640 1 -- 172.21.15.102:0/2657570858 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56133f0d11f0 con 0x56133f0c2420 2024-04-24T23:25:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.833+0000 7f8224e25640 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0d0730 0x56133f0d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.833+0000 7f821f7fe640 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f13cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:25:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.833+0000 7f821ffff640 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f13c710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f82100ecc50 tx=0x7f82100bb880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7987f853b577d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.833+0000 7f81ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:25:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.833+0000 7f81ff7fe640 1 -- 172.21.15.102:0/2657570858 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82100bc0f0 con 0x56133f0c2420 2024-04-24T23:25:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.833+0000 7f81ff7fe640 1 -- 172.21.15.102:0/2657570858 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7f82100b8060 con 0x56133f0c2420 2024-04-24T23:25:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.833+0000 7f8229592740 1 -- 172.21.15.102:0/2657570858 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56133f02e540 con 0x56133f0c2420 2024-04-24T23:25:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.837+0000 7f81ff7fe640 1 -- 172.21.15.102:0/2657570858 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82100c4660 con 0x56133f0c2420 2024-04-24T23:25:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.842+0000 7f8229592740 10 client.25696 fetch_fsmap learned FSMap version 26 2024-04-24T23:25:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.842+0000 7f8229592740 1 -- 172.21.15.102:0/2657570858 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56133f144260 con 0x56133f0c2420 2024-04-24T23:25:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.844+0000 7f81ff7fe640 1 -- 172.21.15.102:0/2657570858 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56133f02e540 con 0x56133f0c2420 2024-04-24T23:25:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.844+0000 7f81ff7fe640 1 -- 172.21.15.102:0/2657570858 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f82100ecd00 con 0x56133f0c2420 2024-04-24T23:25:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.845+0000 7f8229592740 10 client.25696 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:25:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.845+0000 7f8229592740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:25:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.845+0000 7f8229592740 20 client.25696 populate_metadata read hostname 'smithi102' 2024-04-24T23:25:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.845+0000 7f8229592740 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56133f1447d0 0x56133f146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.845+0000 7f8229592740 4 client.25696 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:25:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.845+0000 7f8229592740 1 -- 172.21.15.102:0/2657570858 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56133efac8e0 con 0x56133f1447d0 2024-04-24T23:25:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.847+0000 7f821f7fe640 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56133f1447d0 0x56133f146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.860+0000 7f821f7fe640 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56133f1447d0 0x56133f146bc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d43fcd1a041103 server_cookie=dcde677ea7435ac2 in_seq=0 out_seq=0 2024-04-24T23:25:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.860+0000 7f81ff7fe640 10 client.25696 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.889+0000 7f81ff7fe640 1 -- 172.21.15.102:0/2657570858 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56133efac8e0 con 0x56133f1447d0 2024-04-24T23:25:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.890+0000 7f81ff7fe640 10 client.25696 handle_command_reply: tid=0 2024-04-24T23:25:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.890+0000 7f8229592740 1 client.25696 shutdown 2024-04-24T23:25:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.891+0000 7f8229592740 1 -- 172.21.15.102:0/2657570858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 msgr2=0x56133f13c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.891+0000 7f8229592740 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f13c710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f82100ecc50 tx=0x7f82100bb880 comp rx=0 tx=0).stop 2024-04-24T23:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.893+0000 7f8229592740 1 -- 172.21.15.102:0/2657570858 shutdown_connections 2024-04-24T23:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.893+0000 7f8229592740 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56133f0d0730 0x56133f0d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.893+0000 7f8229592740 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x56133f1447d0 0x56133f146bc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.893+0000 7f8229592740 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56133f0cd410 0x56133f13cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.893+0000 7f8229592740 1 --2- 172.21.15.102:0/2657570858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56133f0c2420 0x56133f13c710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.893+0000 7f8229592740 1 -- 172.21.15.102:0/2657570858 >> 172.21.15.102:0/2657570858 conn(0x56133f0be740 msgr2=0x56133f0c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.894+0000 7f81ff7fe640 0 client.25696 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.894+0000 7f8229592740 1 -- 172.21.15.102:0/2657570858 shutdown_connections 2024-04-24T23:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.894+0000 7f8229592740 1 -- 172.21.15.102:0/2657570858 wait complete. 2024-04-24T23:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.894+0000 7f8229592740 20 client.25696 trim_cache size 0 max 16384 2024-04-24T23:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:25:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.898+0000 7f8225626640 1 -- 172.21.15.102:0/2787943039 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82080a2d70 msgr2=0x7f82080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.898+0000 7f8225626640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82080a2d70 0x7f82080a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82180133b0 tx=0x7f821800c040 comp rx=0 tx=0).stop 2024-04-24T23:25:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.898+0000 7f8225626640 1 -- 172.21.15.102:0/2787943039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201089b0 msgr2=0x7f8220122550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.898+0000 7f8225626640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201089b0 0x7f8220122550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8210000c00 tx=0x7f8210005270 comp rx=0 tx=0).stop 2024-04-24T23:25:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.899+0000 7f8225626640 1 -- 172.21.15.102:0/2787943039 shutdown_connections 2024-04-24T23:25:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.899+0000 7f8225626640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f82080a2d70 0x7f82080a5230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.899+0000 7f8225626640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82201140f0 0x7f8220114580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.899+0000 7f8225626640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f822010d2c0 0x7f8220122a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.899+0000 7f8225626640 1 --2- 172.21.15.102:0/2787943039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201089b0 0x7f8220122550 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.899+0000 7f8225626640 1 -- 172.21.15.102:0/2787943039 >> 172.21.15.102:0/2787943039 conn(0x7f822006c680 msgr2=0x7f8220118d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.899+0000 7f8225626640 1 -- 172.21.15.102:0/2787943039 shutdown_connections 2024-04-24T23:25:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:25:50.899+0000 7f8225626640 1 -- 172.21.15.102:0/2787943039 wait complete. 2024-04-24T23:25:51.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:51 smithi183 ceph-mon[33137]: pgmap v399: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 26 MiB/s wr, 6.63k op/s 2024-04-24T23:25:51.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:51 smithi102 ceph-mon[29216]: pgmap v399: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 26 MiB/s wr, 6.63k op/s 2024-04-24T23:25:51.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:51 smithi161 ceph-mon[32709]: pgmap v399: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 26 MiB/s wr, 6.63k op/s 2024-04-24T23:25:53.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:53 smithi183 ceph-mon[33137]: pgmap v400: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 19 MiB/s wr, 4.93k op/s 2024-04-24T23:25:53.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:53 smithi102 ceph-mon[29216]: pgmap v400: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 19 MiB/s wr, 4.93k op/s 2024-04-24T23:25:53.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:53 smithi161 ceph-mon[32709]: pgmap v400: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 19 MiB/s wr, 4.93k op/s 2024-04-24T23:25:55.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:55 smithi183 ceph-mon[33137]: pgmap v401: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 25 MiB/s wr, 6.26k op/s 2024-04-24T23:25:55.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:55 smithi102 ceph-mon[29216]: pgmap v401: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 25 MiB/s wr, 6.26k op/s 2024-04-24T23:25:55.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:55 smithi161 ceph-mon[32709]: pgmap v401: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 25 MiB/s wr, 6.26k op/s 2024-04-24T23:25:57.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:57 smithi183 ceph-mon[33137]: pgmap v402: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 21 MiB/s wr, 5.42k op/s 2024-04-24T23:25:57.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:57 smithi102 ceph-mon[29216]: pgmap v402: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 21 MiB/s wr, 5.42k op/s 2024-04-24T23:25:57.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:57 smithi161 ceph-mon[32709]: pgmap v402: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 21 MiB/s wr, 5.42k op/s 2024-04-24T23:25:59.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:25:59 smithi183 ceph-mon[33137]: pgmap v403: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2024-04-24T23:25:59.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:25:59 smithi102 ceph-mon[29216]: pgmap v403: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2024-04-24T23:25:59.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:25:59 smithi161 ceph-mon[32709]: pgmap v403: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2024-04-24T23:26:01.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:01 smithi183 ceph-mon[33137]: pgmap v404: 97 pgs: 97 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 26 MiB/s wr, 6.71k op/s 2024-04-24T23:26:01.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:01 smithi102 ceph-mon[29216]: pgmap v404: 97 pgs: 97 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 26 MiB/s wr, 6.71k op/s 2024-04-24T23:26:01.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:01 smithi161 ceph-mon[32709]: pgmap v404: 97 pgs: 97 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 26 MiB/s wr, 6.71k op/s 2024-04-24T23:26:03.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:03 smithi183 ceph-mon[33137]: pgmap v405: 97 pgs: 97 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 20 MiB/s wr, 4.97k op/s 2024-04-24T23:26:03.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:03 smithi102 ceph-mon[29216]: pgmap v405: 97 pgs: 97 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 20 MiB/s wr, 4.97k op/s 2024-04-24T23:26:03.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:03 smithi161 ceph-mon[32709]: pgmap v405: 97 pgs: 97 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 20 MiB/s wr, 4.97k op/s 2024-04-24T23:26:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.126+0000 7fa1323b7640 1 -- 172.21.15.102:0/1306555093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c10e4b0 msgr2=0x7fa12c108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.126+0000 7fa1323b7640 1 --2- 172.21.15.102:0/1306555093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c10e4b0 0x7fa12c108740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa124009ca0 tx=0x7fa12402a8f0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.126+0000 7fa1323b7640 1 -- 172.21.15.102:0/1306555093 shutdown_connections 2024-04-24T23:26:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.126+0000 7fa1323b7640 1 --2- 172.21.15.102:0/1306555093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c10e4b0 0x7fa12c108740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.126+0000 7fa1323b7640 1 --2- 172.21.15.102:0/1306555093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa12c10da10 0x7fa12c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.126+0000 7fa1323b7640 1 --2- 172.21.15.102:0/1306555093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa12c10fcb0 0x7fa12c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.126+0000 7fa1323b7640 1 -- 172.21.15.102:0/1306555093 >> 172.21.15.102:0/1306555093 conn(0x7fa12c06c6f0 msgr2=0x7fa12c06cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.133+0000 7fa1323b7640 1 -- 172.21.15.102:0/1306555093 shutdown_connections 2024-04-24T23:26:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.136+0000 7fa1323b7640 1 -- 172.21.15.102:0/1306555093 wait complete. 2024-04-24T23:26:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.136+0000 7fa1323b7640 1 Processor -- start 2024-04-24T23:26:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.137+0000 7fa1323b7640 1 -- start start 2024-04-24T23:26:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.137+0000 7fa1323b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa12c10da10 0x7fa12c11e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.137+0000 7fa1323b7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa12c10e4b0 0x7fa12c118710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.137+0000 7fa1323b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c10fcb0 0x7fa12c118c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.137+0000 7fa1323b7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c111000 con 0x7fa12c10e4b0 2024-04-24T23:26:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.137+0000 7fa1323b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c110e80 con 0x7fa12c10fcb0 2024-04-24T23:26:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.137+0000 7fa1323b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c111180 con 0x7fa12c10da10 2024-04-24T23:26:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.138+0000 7fa12bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa12c10da10 0x7fa12c11e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.138+0000 7fa12bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa12c10da10 0x7fa12c11e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:44888/0 (socket says 172.21.15.102:44888) 2024-04-24T23:26:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.138+0000 7fa12bfff640 1 -- 172.21.15.102:0/550231010 learned_addr learned my addr 172.21.15.102:0/550231010 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.138+0000 7fa130954640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c10fcb0 0x7fa12c118c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.139+0000 7fa12b7fe640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa12c10e4b0 0x7fa12c118710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.139+0000 7fa130954640 1 -- 172.21.15.102:0/550231010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa12c10da10 msgr2=0x7fa12c11e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.139+0000 7fa130954640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa12c10da10 0x7fa12c11e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.139+0000 7fa130954640 1 -- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa12c10e4b0 msgr2=0x7fa12c118710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.139+0000 7fa130954640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa12c10e4b0 0x7fa12c118710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.139+0000 7fa130954640 1 -- 172.21.15.102:0/550231010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c1193d0 con 0x7fa12c10fcb0 2024-04-24T23:26:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.141+0000 7fa130954640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c10fcb0 0x7fa12c118c50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa12402aee0 tx=0x7fa124030890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.145+0000 7fa1297fa640 1 -- 172.21.15.102:0/550231010 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124003c30 con 0x7fa12c10fcb0 2024-04-24T23:26:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.145+0000 7fa1323b7640 1 -- 172.21.15.102:0/550231010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c1bbb50 con 0x7fa12c10fcb0 2024-04-24T23:26:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.145+0000 7fa1323b7640 1 -- 172.21.15.102:0/550231010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c1bbfb0 con 0x7fa12c10fcb0 2024-04-24T23:26:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.145+0000 7fa1297fa640 1 -- 172.21.15.102:0/550231010 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa12402d070 con 0x7fa12c10fcb0 2024-04-24T23:26:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.145+0000 7fa1297fa640 1 -- 172.21.15.102:0/550231010 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1240047e0 con 0x7fa12c10fcb0 2024-04-24T23:26:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.149+0000 7fa1351a2740 1 Processor -- start 2024-04-24T23:26:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.150+0000 7fa1351a2740 1 -- start start 2024-04-24T23:26:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.150+0000 7fa1297fa640 1 -- 172.21.15.102:0/550231010 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fa12404d430 con 0x7fa12c10fcb0 2024-04-24T23:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.151+0000 7fa1297fa640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa1140a2f00 0x7fa1140a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.151+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 0x55c2ed220790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.151+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed220cd0 0x55c2ed22ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.151+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.151+0000 7fa1351a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c2ed184cb0 con 0x55c2ed2203b0 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.151+0000 7fa1351a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2ed12d5b0 con 0x55c2ed220cd0 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.151+0000 7fa1351a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2ed180c70 con 0x55c2ed22b3a0 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa130954640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa130954640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:44902/0 (socket says 172.21.15.102:44902) 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa130954640 1 -- 172.21.15.102:0/4256659922 learned_addr learned my addr 172.21.15.102:0/4256659922 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa12b7fe640 1 --2- 172.21.15.102:0/4256659922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed220cd0 0x55c2ed22ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa1297fa640 1 -- 172.21.15.102:0/550231010 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7fa1240f1720 con 0x7fa12c10fcb0 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa130954640 1 -- 172.21.15.102:0/4256659922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed220cd0 msgr2=0x55c2ed22ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa130954640 1 --2- 172.21.15.102:0/4256659922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed220cd0 0x55c2ed22ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa130954640 1 -- 172.21.15.102:0/4256659922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 msgr2=0x55c2ed220790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa130954640 1 --2- 172.21.15.102:0/4256659922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 0x55c2ed220790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.152+0000 7fa130954640 1 -- 172.21.15.102:0/4256659922 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ece99210 con 0x55c2ed22b3a0 2024-04-24T23:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.154+0000 7fa130954640 1 --2- 172.21.15.102:0/4256659922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22d790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa124009ca0 tx=0x7fa1240095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5597321f1c91929 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.154+0000 7fa1127fc640 1 -- 172.21.15.102:0/4256659922 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12409b350 con 0x55c2ed22b3a0 2024-04-24T23:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.154+0000 7fa12b7fe640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa1140a2f00 0x7fa1140a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.154+0000 7fa1127fc640 1 -- 172.21.15.102:0/4256659922 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa124036030 con 0x55c2ed22b3a0 2024-04-24T23:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.154+0000 7fa1127fc640 1 -- 172.21.15.102:0/4256659922 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12409b170 con 0x55c2ed22b3a0 2024-04-24T23:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.155+0000 7fa1351a2740 1 -- 172.21.15.102:0/4256659922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 msgr2=0x55c2ed22d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.155+0000 7fa1351a2740 1 --2- 172.21.15.102:0/4256659922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22d790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa124009ca0 tx=0x7fa1240095c0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.155+0000 7fa12b7fe640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa1140a2f00 0x7fa1140a53c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa120006de0 tx=0x7fa120004700 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.155+0000 7fa1351a2740 1 -- 172.21.15.102:0/4256659922 shutdown_connections 2024-04-24T23:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.155+0000 7fa1351a2740 1 --2- 172.21.15.102:0/4256659922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22d790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.155+0000 7fa1351a2740 1 --2- 172.21.15.102:0/4256659922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed220cd0 0x55c2ed22ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.155+0000 7fa1351a2740 1 --2- 172.21.15.102:0/4256659922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 0x55c2ed220790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.155+0000 7fa1351a2740 1 -- 172.21.15.102:0/4256659922 >> 172.21.15.102:0/4256659922 conn(0x55c2ed21c6d0 msgr2=0x55c2ed21cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.156+0000 7fa1351a2740 1 -- 172.21.15.102:0/4256659922 shutdown_connections 2024-04-24T23:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.156+0000 7fa1351a2740 1 -- 172.21.15.102:0/4256659922 wait complete. 2024-04-24T23:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.156+0000 7fa1351a2740 1 -- start start 2024-04-24T23:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.156+0000 7fa1351a2740 1 Processor -- start 2024-04-24T23:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.157+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 0x55c2ed228300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.157+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed220cd0 0x55c2ed228840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.157+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 0x55c2ed228d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.157+0000 7fa1351a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c2ed22df50 con 0x55c2ed2203b0 2024-04-24T23:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.157+0000 7fa1351a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2ed140b50 con 0x55c2ed22b3a0 2024-04-24T23:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.157+0000 7fa1351a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2ed22e0d0 con 0x55c2ed220cd0 2024-04-24T23:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.157+0000 7fa1351a2740 10 client.0 _ll_register_callbacks cb 0x55c2ed17bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.158+0000 7fa12bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 0x55c2ed228300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.158+0000 7fa12bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 0x55c2ed228300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41714/0 (socket says 172.21.15.102:41714) 2024-04-24T23:26:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.158+0000 7fa12bfff640 1 -- 172.21.15.102:0/3324858203 learned_addr learned my addr 172.21.15.102:0/3324858203 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.159+0000 7fa130954640 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 0x55c2ed228d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.160+0000 7fa12bfff640 1 -- 172.21.15.102:0/3324858203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed220cd0 msgr2=0x55c2ed228840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.160+0000 7fa12bfff640 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed220cd0 0x55c2ed228840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.160+0000 7fa12bfff640 1 -- 172.21.15.102:0/3324858203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 msgr2=0x55c2ed228d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.160+0000 7fa12bfff640 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 0x55c2ed228d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.160+0000 7fa12bfff640 1 -- 172.21.15.102:0/3324858203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ecf65620 con 0x55c2ed2203b0 2024-04-24T23:26:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.165+0000 7fa12bfff640 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 0x55c2ed228300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00d210 tx=0x7fa11c00d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488feb579f091ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.169+0000 7fa10b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:26:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.173+0000 7fa10b7fe640 1 -- 172.21.15.102:0/3324858203 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0125f0 con 0x55c2ed2203b0 2024-04-24T23:26:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.173+0000 7fa1351a2740 1 -- 172.21.15.102:0/3324858203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa124036030 con 0x55c2ed2203b0 2024-04-24T23:26:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.173+0000 7fa10b7fe640 1 -- 172.21.15.102:0/3324858203 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7fa11c012790 con 0x55c2ed2203b0 2024-04-24T23:26:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.180+0000 7fa10b7fe640 1 -- 172.21.15.102:0/3324858203 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa11c021050 con 0x55c2ed2203b0 2024-04-24T23:26:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.180+0000 7fa1351a2740 10 client.16365 fetch_fsmap learned FSMap version 26 2024-04-24T23:26:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.180+0000 7fa1351a2740 1 -- 172.21.15.102:0/3324858203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2ed23a140 con 0x55c2ed2203b0 2024-04-24T23:26:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.184+0000 7fa10b7fe640 1 -- 172.21.15.102:0/3324858203 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa124036030 con 0x55c2ed2203b0 2024-04-24T23:26:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.185+0000 7fa10b7fe640 1 -- 172.21.15.102:0/3324858203 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa11c013a70 con 0x55c2ed2203b0 2024-04-24T23:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.186+0000 7fa1351a2740 10 client.16365 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.186+0000 7fa1351a2740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.186+0000 7fa1351a2740 20 client.16365 populate_metadata read hostname 'smithi102' 2024-04-24T23:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.186+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55c2ed23baa0 0x55c2ed23df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.186+0000 7fa1351a2740 4 client.16365 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.186+0000 7fa1351a2740 1 -- 172.21.15.102:0/3324858203 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2ed1921f0 con 0x55c2ed23baa0 2024-04-24T23:26:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.191+0000 7fa12b7fe640 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55c2ed23baa0 0x55c2ed23df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.192+0000 7fa12b7fe640 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55c2ed23baa0 0x55c2ed23df40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d57513ec9ed1d1 server_cookie=fadfddc993380c37 in_seq=0 out_seq=0 2024-04-24T23:26:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.192+0000 7fa10b7fe640 10 client.16365 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.193+0000 7fa10b7fe640 1 -- 172.21.15.102:0/3324858203 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2ed1921f0 con 0x55c2ed23baa0 2024-04-24T23:26:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.193+0000 7fa10b7fe640 10 client.16365 handle_command_reply: tid=0 2024-04-24T23:26:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.193+0000 7fa1351a2740 1 client.16365 shutdown 2024-04-24T23:26:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.205+0000 7fa1351a2740 1 -- 172.21.15.102:0/3324858203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 msgr2=0x55c2ed228300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.205+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 0x55c2ed228300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00d210 tx=0x7fa11c00d6d0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.205+0000 7fa1351a2740 1 -- 172.21.15.102:0/3324858203 shutdown_connections 2024-04-24T23:26:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.205+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 0x55c2ed228d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.205+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed220cd0 0x55c2ed228840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.205+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55c2ed23baa0 0x55c2ed23df40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.205+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3324858203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed2203b0 0x55c2ed228300 secure :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00d210 tx=0x7fa11c00d6d0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.205+0000 7fa1351a2740 1 -- 172.21.15.102:0/3324858203 >> 172.21.15.102:0/3324858203 conn(0x55c2ed21c6d0 msgr2=0x55c2ed2223d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.205+0000 7fa10b7fe640 0 client.16365 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.211+0000 7fa1351a2740 1 -- 172.21.15.102:0/3324858203 shutdown_connections 2024-04-24T23:26:04.217 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.213+0000 7fa1351a2740 1 -- 172.21.15.102:0/3324858203 wait complete. 2024-04-24T23:26:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.213+0000 7fa1351a2740 20 client.16365 trim_cache size 0 max 16384 2024-04-24T23:26:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.267+0000 7fa1351a2740 1 Processor -- start 2024-04-24T23:26:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.267+0000 7fa1351a2740 1 -- start start 2024-04-24T23:26:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.267+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed236e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.267+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.268+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed243810 0x55c2ed243c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.268+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.268+0000 7fa12b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:44924/0 (socket says 172.21.15.102:44924) 2024-04-24T23:26:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.268+0000 7fa12b7fe640 1 -- 172.21.15.102:0/3143482753 learned_addr learned my addr 172.21.15.102:0/3143482753 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.268+0000 7fa12bfff640 1 --2- 172.21.15.102:0/3143482753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed236e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.268+0000 7fa130954640 1 --2- 172.21.15.102:0/3143482753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed243810 0x55c2ed243c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.268+0000 7fa1351a2740 1 -- 172.21.15.102:0/3143482753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa1180110b0 con 0x55c2ed220cd0 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.268+0000 7fa1351a2740 1 -- 172.21.15.102:0/3143482753 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa11800e980 con 0x55c2ed243810 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.268+0000 7fa1351a2740 1 -- 172.21.15.102:0/3143482753 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa11800c880 con 0x55c2ed22b3a0 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.269+0000 7fa12bfff640 1 -- 172.21.15.102:0/3143482753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 msgr2=0x55c2ed22aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.269+0000 7fa12bfff640 1 --2- 172.21.15.102:0/3143482753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.269+0000 7fa12bfff640 1 -- 172.21.15.102:0/3143482753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed243810 msgr2=0x55c2ed243c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.269+0000 7fa12bfff640 1 --2- 172.21.15.102:0/3143482753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed243810 0x55c2ed243c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.269+0000 7fa12bfff640 1 -- 172.21.15.102:0/3143482753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2ed226850 con 0x55c2ed220cd0 2024-04-24T23:26:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.270+0000 7fa12bfff640 1 --2- 172.21.15.102:0/3143482753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed236e10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa11c011000 tx=0x7fa11c006e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd6732612ed6fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.270+0000 7fa111ffb640 1 -- 172.21.15.102:0/3143482753 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c02b020 con 0x55c2ed220cd0 2024-04-24T23:26:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.270+0000 7fa111ffb640 1 -- 172.21.15.102:0/3143482753 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa11c018070 con 0x55c2ed220cd0 2024-04-24T23:26:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.270+0000 7fa111ffb640 1 -- 172.21.15.102:0/3143482753 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c013480 con 0x55c2ed220cd0 2024-04-24T23:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.271+0000 7fa1351a2740 1 -- 172.21.15.102:0/3143482753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 msgr2=0x55c2ed236e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.271+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3143482753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed236e10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa11c011000 tx=0x7fa11c006e30 comp rx=0 tx=0).stop 2024-04-24T23:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.271+0000 7fa1351a2740 1 -- 172.21.15.102:0/3143482753 shutdown_connections 2024-04-24T23:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.272+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3143482753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed243810 0x55c2ed243c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.272+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3143482753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.272+0000 7fa1351a2740 1 --2- 172.21.15.102:0/3143482753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed236e10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.272+0000 7fa1351a2740 1 -- 172.21.15.102:0/3143482753 >> 172.21.15.102:0/3143482753 conn(0x55c2ed21c6d0 msgr2=0x55c2ed2172d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.272+0000 7fa1351a2740 1 -- 172.21.15.102:0/3143482753 shutdown_connections 2024-04-24T23:26:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.272+0000 7fa1351a2740 1 -- 172.21.15.102:0/3143482753 wait complete. 2024-04-24T23:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.272+0000 7fa1351a2740 1 -- start start 2024-04-24T23:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa1351a2740 1 Processor -- start 2024-04-24T23:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed233570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa1351a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed243810 0x55c2ed22ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa1351a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa11800ce40 con 0x55c2ed220cd0 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa1351a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa11800e2b0 con 0x55c2ed22b3a0 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa1351a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa11800d510 con 0x55c2ed243810 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa1351a2740 10 client.0 _ll_register_callbacks cb 0x7fa11c0071e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa12bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed233570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa12bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed233570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41730/0 (socket says 172.21.15.102:41730) 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa12bfff640 1 -- 172.21.15.102:0/275477247 learned_addr learned my addr 172.21.15.102:0/275477247 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa12b7fe640 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa130954640 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed243810 0x55c2ed22ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa12bfff640 1 -- 172.21.15.102:0/275477247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed243810 msgr2=0x55c2ed22ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa12bfff640 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed243810 0x55c2ed22ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa12bfff640 1 -- 172.21.15.102:0/275477247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 msgr2=0x55c2ed22e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa12bfff640 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.273+0000 7fa12bfff640 1 -- 172.21.15.102:0/275477247 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ed22f490 con 0x55c2ed220cd0 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.274+0000 7fa12bfff640 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed233570 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00dbd0 tx=0x7fa11c0136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcc1b1042ddd16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.274+0000 7fa10b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.274+0000 7fa10b7fe640 1 -- 172.21.15.102:0/275477247 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c035060 con 0x55c2ed220cd0 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.274+0000 7fa10b7fe640 1 -- 172.21.15.102:0/275477247 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8579+0+0 (secure 0 0 0) 0x7fa11c01c480 con 0x55c2ed220cd0 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.274+0000 7fa1351a2740 1 -- 172.21.15.102:0/275477247 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11c018070 con 0x55c2ed220cd0 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.274+0000 7fa10b7fe640 1 -- 172.21.15.102:0/275477247 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa11c02b030 con 0x55c2ed220cd0 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.274+0000 7fa1351a2740 10 client.16371 fetch_fsmap learned FSMap version 26 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.274+0000 7fa1351a2740 1 -- 172.21.15.102:0/275477247 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2ed2a1f20 con 0x55c2ed220cd0 2024-04-24T23:26:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa10b7fe640 1 -- 172.21.15.102:0/275477247 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa11c018070 con 0x55c2ed220cd0 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa10b7fe640 1 -- 172.21.15.102:0/275477247 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa11c01ca30 con 0x55c2ed220cd0 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa1351a2740 10 client.16371 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa1351a2740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa1351a2740 20 client.16371 populate_metadata read hostname 'smithi102' 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa1351a2740 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55c2ed2a2580 0x55c2ed2a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa1351a2740 4 client.16371 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa1351a2740 1 -- 172.21.15.102:0/275477247 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2ed18c540 con 0x55c2ed2a2580 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa12b7fe640 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55c2ed2a2580 0x55c2ed2a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.276+0000 7fa10b7fe640 10 client.16371 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.275+0000 7fa12b7fe640 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55c2ed2a2580 0x55c2ed2a49a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a247a7919379e08 server_cookie=bd52fda331cacb27 in_seq=0 out_seq=0 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa10b7fe640 1 -- 172.21.15.102:0/275477247 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2ed18c540 con 0x55c2ed2a2580 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa10b7fe640 10 client.16371 handle_command_reply: tid=0 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 client.16371 shutdown 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 -- 172.21.15.102:0/275477247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 msgr2=0x55c2ed233570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed233570 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00dbd0 tx=0x7fa11c0136b0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 -- 172.21.15.102:0/275477247 shutdown_connections 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2ed243810 0x55c2ed22ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2ed22b3a0 0x55c2ed22e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55c2ed2a2580 0x55c2ed2a49a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 --2- 172.21.15.102:0/275477247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2ed220cd0 0x55c2ed233570 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 -- 172.21.15.102:0/275477247 >> 172.21.15.102:0/275477247 conn(0x55c2ed21c6d0 msgr2=0x55c2ed2235e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa10b7fe640 0 client.16371 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 -- 172.21.15.102:0/275477247 shutdown_connections 2024-04-24T23:26:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.732+0000 7fa1351a2740 1 -- 172.21.15.102:0/275477247 wait complete. 2024-04-24T23:26:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.733+0000 7fa1351a2740 20 client.16371 trim_cache size 0 max 16384 2024-04-24T23:26:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:26:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:26:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:26:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:26:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.741+0000 7fa1323b7640 1 -- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa1140a2f00 msgr2=0x7fa1140a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.741+0000 7fa1323b7640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa1140a2f00 0x7fa1140a53c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa120006de0 tx=0x7fa120004700 comp rx=0 tx=0).stop 2024-04-24T23:26:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.741+0000 7fa1323b7640 1 -- 172.21.15.102:0/550231010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c10fcb0 msgr2=0x7fa12c118c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.741+0000 7fa1323b7640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c10fcb0 0x7fa12c118c50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa12402aee0 tx=0x7fa124030890 comp rx=0 tx=0).stop 2024-04-24T23:26:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.746+0000 7fa1323b7640 1 -- 172.21.15.102:0/550231010 shutdown_connections 2024-04-24T23:26:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.749+0000 7fa1323b7640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12c10fcb0 0x7fa12c118c50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.749+0000 7fa1323b7640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fa1140a2f00 0x7fa1140a53c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.749+0000 7fa1323b7640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa12c10e4b0 0x7fa12c118710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.749+0000 7fa1323b7640 1 --2- 172.21.15.102:0/550231010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa12c10da10 0x7fa12c11e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.749+0000 7fa1323b7640 1 -- 172.21.15.102:0/550231010 >> 172.21.15.102:0/550231010 conn(0x7fa12c06c6f0 msgr2=0x7fa12c109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.751+0000 7fa1323b7640 1 -- 172.21.15.102:0/550231010 shutdown_connections 2024-04-24T23:26:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:04.751+0000 7fa1323b7640 1 -- 172.21.15.102:0/550231010 wait complete. 2024-04-24T23:26:05.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:05 smithi183 ceph-mon[33137]: pgmap v406: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 23 MiB/s wr, 5.95k op/s 2024-04-24T23:26:05.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:05 smithi102 ceph-mon[29216]: pgmap v406: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 23 MiB/s wr, 5.95k op/s 2024-04-24T23:26:05.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:05 smithi161 ceph-mon[32709]: pgmap v406: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 23 MiB/s wr, 5.95k op/s 2024-04-24T23:26:07.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:07 smithi183 ceph-mon[33137]: pgmap v407: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2024-04-24T23:26:07.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:07 smithi183 ceph-mon[33137]: osdmap e104: 12 total, 12 up, 12 in 2024-04-24T23:26:07.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:07 smithi102 ceph-mon[29216]: pgmap v407: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2024-04-24T23:26:07.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:07 smithi102 ceph-mon[29216]: osdmap e104: 12 total, 12 up, 12 in 2024-04-24T23:26:07.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:07 smithi161 ceph-mon[32709]: pgmap v407: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2024-04-24T23:26:07.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:07 smithi161 ceph-mon[32709]: osdmap e104: 12 total, 12 up, 12 in 2024-04-24T23:26:09.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:09 smithi183 ceph-mon[33137]: pgmap v409: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 19 MiB/s wr, 4.91k op/s 2024-04-24T23:26:09.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:09 smithi102 ceph-mon[29216]: pgmap v409: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 19 MiB/s wr, 4.91k op/s 2024-04-24T23:26:09.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:09 smithi161 ceph-mon[32709]: pgmap v409: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 19 MiB/s wr, 4.91k op/s 2024-04-24T23:26:11.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:11 smithi183 ceph-mon[33137]: pgmap v410: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2024-04-24T23:26:11.824 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:11 smithi102 ceph-mon[29216]: pgmap v410: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2024-04-24T23:26:11.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:11 smithi161 ceph-mon[32709]: pgmap v410: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2024-04-24T23:26:13.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:13 smithi183 ceph-mon[33137]: pgmap v411: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2024-04-24T23:26:13.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:13 smithi102 ceph-mon[29216]: pgmap v411: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2024-04-24T23:26:13.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:13 smithi161 ceph-mon[32709]: pgmap v411: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2024-04-24T23:26:15.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:15 smithi183 ceph-mon[33137]: pgmap v412: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 19 MiB/s wr, 4.75k op/s 2024-04-24T23:26:15.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:15 smithi102 ceph-mon[29216]: pgmap v412: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 19 MiB/s wr, 4.75k op/s 2024-04-24T23:26:15.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:15 smithi161 ceph-mon[32709]: pgmap v412: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 19 MiB/s wr, 4.75k op/s 2024-04-24T23:26:17.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:17 smithi183 ceph-mon[33137]: pgmap v413: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 19 MiB/s wr, 4.74k op/s 2024-04-24T23:26:17.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:17 smithi102 ceph-mon[29216]: pgmap v413: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 19 MiB/s wr, 4.74k op/s 2024-04-24T23:26:17.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:17 smithi161 ceph-mon[32709]: pgmap v413: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 19 MiB/s wr, 4.74k op/s 2024-04-24T23:26:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.034+0000 7fce2f59e640 1 -- 172.21.15.102:0/1390927737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce3010e410 msgr2=0x7fce3011cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.034+0000 7fce2f59e640 1 --2- 172.21.15.102:0/1390927737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce3010e410 0x7fce3011cf30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fce20009900 tx=0x7fce2002a4d0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.041+0000 7fce2f59e640 1 -- 172.21.15.102:0/1390927737 shutdown_connections 2024-04-24T23:26:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.041+0000 7fce2f59e640 1 --2- 172.21.15.102:0/1390927737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce3010e410 0x7fce3011cf30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.041+0000 7fce2f59e640 1 --2- 172.21.15.102:0/1390927737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce301101e0 0x7fce3010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.041+0000 7fce2f59e640 1 --2- 172.21.15.102:0/1390927737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce3010fe00 0x7fce3010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.041+0000 7fce2f59e640 1 -- 172.21.15.102:0/1390927737 >> 172.21.15.102:0/1390927737 conn(0x7fce3006c680 msgr2=0x7fce3006ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.042+0000 7fce2f59e640 1 -- 172.21.15.102:0/1390927737 shutdown_connections 2024-04-24T23:26:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.042+0000 7fce2f59e640 1 -- 172.21.15.102:0/1390927737 wait complete. 2024-04-24T23:26:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.043+0000 7fce2f59e640 1 Processor -- start 2024-04-24T23:26:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.045+0000 7fce2f59e640 1 -- start start 2024-04-24T23:26:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.045+0000 7fce2f59e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce3010e410 0x7fce3011c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.045+0000 7fce2f59e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3010fe00 0x7fce30115640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.045+0000 7fce2f59e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce301101e0 0x7fce30115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.045+0000 7fce2f59e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce301207b0 con 0x7fce3010e410 2024-04-24T23:26:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.045+0000 7fce2f59e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fce30120630 con 0x7fce3010fe00 2024-04-24T23:26:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.045+0000 7fce2f59e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce30120930 con 0x7fce301101e0 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.049+0000 7fce2ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce301101e0 0x7fce30115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.049+0000 7fce2ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce301101e0 0x7fce30115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:56440/0 (socket says 172.21.15.102:56440) 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.049+0000 7fce2ed9d640 1 -- 172.21.15.102:0/2909159156 learned_addr learned my addr 172.21.15.102:0/2909159156 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.050+0000 7fce2ed9d640 1 -- 172.21.15.102:0/2909159156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3010fe00 msgr2=0x7fce30115640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.050+0000 7fce2ed9d640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3010fe00 0x7fce30115640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.050+0000 7fce2ed9d640 1 -- 172.21.15.102:0/2909159156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce3010e410 msgr2=0x7fce3011c5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.050+0000 7fce2ed9d640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce3010e410 0x7fce3011c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.050+0000 7fce2ed9d640 1 -- 172.21.15.102:0/2909159156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce301161f0 con 0x7fce301101e0 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.051+0000 7fce2ed9d640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce301101e0 0x7fce30115b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fce2002a9f0 tx=0x7fce20003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.051+0000 7fce1f7fe640 1 -- 172.21.15.102:0/2909159156 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2003f070 con 0x7fce301101e0 2024-04-24T23:26:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.051+0000 7fce1f7fe640 1 -- 172.21.15.102:0/2909159156 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce2003bd20 con 0x7fce301101e0 2024-04-24T23:26:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.051+0000 7fce1f7fe640 1 -- 172.21.15.102:0/2909159156 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20035030 con 0x7fce301101e0 2024-04-24T23:26:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.051+0000 7fce2f59e640 1 -- 172.21.15.102:0/2909159156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce30116480 con 0x7fce301101e0 2024-04-24T23:26:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.051+0000 7fce2f59e640 1 -- 172.21.15.102:0/2909159156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3011a3b0 con 0x7fce301101e0 2024-04-24T23:26:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.056+0000 7fce3781b740 1 Processor -- start 2024-04-24T23:26:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.058+0000 7fce1f7fe640 1 -- 172.21.15.102:0/2909159156 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fce20042070 con 0x7fce301101e0 2024-04-24T23:26:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.062+0000 7fce3781b740 1 -- start start 2024-04-24T23:26:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.062+0000 7fce3781b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b22c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.062+0000 7fce3781b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b236ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.062+0000 7fce3781b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b239800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.062+0000 7fce3781b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55682b190cb0 con 0x55682b22c420 2024-04-24T23:26:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.062+0000 7fce3781b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55682b1395b0 con 0x55682b237410 2024-04-24T23:26:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.062+0000 7fce3781b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55682b18cc70 con 0x55682b22cd40 2024-04-24T23:26:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.063+0000 7fce1f7fe640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fce180a2df0 0x7fce180a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.063+0000 7fce1f7fe640 1 -- 172.21.15.102:0/2909159156 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8623+0+0 (secure 0 0 0) 0x7fce200efde0 con 0x7fce301101e0 2024-04-24T23:26:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.063+0000 7fce2ed9d640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fce180a2df0 0x7fce180a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b22c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b22c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50920/0 (socket says 172.21.15.102:50920) 2024-04-24T23:26:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2dd9b640 1 -- 172.21.15.102:0/850286654 learned_addr learned my addr 172.21.15.102:0/850286654 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/850286654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b239800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2e59c640 1 --2- 172.21.15.102:0/850286654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b236ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2dd9b640 1 -- 172.21.15.102:0/850286654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 msgr2=0x55682b236ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/850286654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b236ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2dd9b640 1 -- 172.21.15.102:0/850286654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 msgr2=0x55682b239800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/850286654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b239800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.066+0000 7fce2dd9b640 1 -- 172.21.15.102:0/850286654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682aea5210 con 0x55682b22c420 2024-04-24T23:26:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.068+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/850286654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b22c800 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fce28012660 tx=0x7fce2800b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e063e5a6fb684dee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.069+0000 7fce2ed9d640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fce180a2df0 0x7fce180a52b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce2003bfd0 tx=0x7fce2002a4d0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.069+0000 7fce1cff9640 1 -- 172.21.15.102:0/850286654 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2800b190 con 0x55682b22c420 2024-04-24T23:26:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.069+0000 7fce1cff9640 1 -- 172.21.15.102:0/850286654 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce280198f0 con 0x55682b22c420 2024-04-24T23:26:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.070+0000 7fce3781b740 1 -- 172.21.15.102:0/850286654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 msgr2=0x55682b22c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.070+0000 7fce3781b740 1 --2- 172.21.15.102:0/850286654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b22c800 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fce28012660 tx=0x7fce2800b600 comp rx=0 tx=0).stop 2024-04-24T23:26:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.070+0000 7fce3781b740 1 -- 172.21.15.102:0/850286654 shutdown_connections 2024-04-24T23:26:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.070+0000 7fce3781b740 1 --2- 172.21.15.102:0/850286654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b239800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.070+0000 7fce3781b740 1 --2- 172.21.15.102:0/850286654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b236ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.070+0000 7fce3781b740 1 --2- 172.21.15.102:0/850286654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b22c800 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.070+0000 7fce3781b740 1 -- 172.21.15.102:0/850286654 >> 172.21.15.102:0/850286654 conn(0x55682b228740 msgr2=0x55682b228b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 -- 172.21.15.102:0/850286654 shutdown_connections 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 -- 172.21.15.102:0/850286654 wait complete. 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 -- start start 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 Processor -- start 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b22c420 0x55682b234310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22cd40 0x55682b234850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b237410 0x55682b234d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55682b23a2a0 con 0x55682b22cd40 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55682b14cb50 con 0x55682b22c420 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.076+0000 7fce3781b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55682b23a420 con 0x55682b237410 2024-04-24T23:26:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.077+0000 7fce3781b740 10 client.0 _ll_register_callbacks cb 0x55682b187ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.078+0000 7fce2e59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22cd40 0x55682b234850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.078+0000 7fce2e59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22cd40 0x55682b234850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50922/0 (socket says 172.21.15.102:50922) 2024-04-24T23:26:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.078+0000 7fce2e59c640 1 -- 172.21.15.102:0/1379964610 learned_addr learned my addr 172.21.15.102:0/1379964610 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.078+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b22c420 0x55682b234310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.078+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b237410 0x55682b234d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.082+0000 7fce2e59c640 1 -- 172.21.15.102:0/1379964610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b237410 msgr2=0x55682b234d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.082+0000 7fce2e59c640 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b237410 0x55682b234d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.082+0000 7fce2e59c640 1 -- 172.21.15.102:0/1379964610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b22c420 msgr2=0x55682b234310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.082+0000 7fce2e59c640 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b22c420 0x55682b234310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.082+0000 7fce2e59c640 1 -- 172.21.15.102:0/1379964610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682af71620 con 0x55682b22cd40 2024-04-24T23:26:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.090+0000 7fce2e59c640 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22cd40 0x55682b234850 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fce2400e1d0 tx=0x7fce2400e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c88248daa41fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.090+0000 7fce15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:26:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.094+0000 7fce15ffb640 1 -- 172.21.15.102:0/1379964610 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24017070 con 0x55682b22cd40 2024-04-24T23:26:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.094+0000 7fce3781b740 1 -- 172.21.15.102:0/1379964610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce280198f0 con 0x55682b22cd40 2024-04-24T23:26:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.094+0000 7fce15ffb640 1 -- 172.21.15.102:0/1379964610 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8623+0+0 (secure 0 0 0) 0x7fce24013680 con 0x55682b22cd40 2024-04-24T23:26:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.098+0000 7fce15ffb640 1 -- 172.21.15.102:0/1379964610 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce24023050 con 0x55682b22cd40 2024-04-24T23:26:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.098+0000 7fce3781b740 10 client.16377 fetch_fsmap learned FSMap version 26 2024-04-24T23:26:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.098+0000 7fce3781b740 1 -- 172.21.15.102:0/1379964610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55682b268520 con 0x55682b22cd40 2024-04-24T23:26:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.100+0000 7fce15ffb640 1 -- 172.21.15.102:0/1379964610 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce280198f0 con 0x55682b22cd40 2024-04-24T23:26:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.101+0000 7fce15ffb640 1 -- 172.21.15.102:0/1379964610 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fce240129b0 con 0x55682b22cd40 2024-04-24T23:26:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.101+0000 7fce3781b740 10 client.16377 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:26:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.101+0000 7fce3781b740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:26:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.101+0000 7fce3781b740 20 client.16377 populate_metadata read hostname 'smithi102' 2024-04-24T23:26:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.101+0000 7fce3781b740 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55682b269d80 0x55682b26c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.101+0000 7fce3781b740 4 client.16377 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.101+0000 7fce3781b740 1 -- 172.21.15.102:0/1379964610 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55682b19e1f0 con 0x55682b269d80 2024-04-24T23:26:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.106+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55682b269d80 0x55682b26c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.117+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55682b269d80 0x55682b26c220 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3960cf4b20104c7b server_cookie=af8ff783eb48d96e in_seq=0 out_seq=0 2024-04-24T23:26:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.119+0000 7fce15ffb640 10 client.16377 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.120+0000 7fce15ffb640 1 -- 172.21.15.102:0/1379964610 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55682b19e1f0 con 0x55682b269d80 2024-04-24T23:26:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.120+0000 7fce15ffb640 10 client.16377 handle_command_reply: tid=0 2024-04-24T23:26:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.120+0000 7fce3781b740 1 client.16377 shutdown 2024-04-24T23:26:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.120+0000 7fce3781b740 1 -- 172.21.15.102:0/1379964610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22cd40 msgr2=0x55682b234850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.120+0000 7fce3781b740 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22cd40 0x55682b234850 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fce2400e1d0 tx=0x7fce2400e690 comp rx=0 tx=0).stop 2024-04-24T23:26:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.125+0000 7fce3781b740 1 -- 172.21.15.102:0/1379964610 shutdown_connections 2024-04-24T23:26:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.125+0000 7fce3781b740 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b237410 0x55682b234d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.125+0000 7fce3781b740 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55682b269d80 0x55682b26c220 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.125+0000 7fce3781b740 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22cd40 0x55682b234850 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.125+0000 7fce3781b740 1 --2- 172.21.15.102:0/1379964610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b22c420 0x55682b234310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.125+0000 7fce3781b740 1 -- 172.21.15.102:0/1379964610 >> 172.21.15.102:0/1379964610 conn(0x55682b228740 msgr2=0x55682b22e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.127+0000 7fce15ffb640 0 client.16377 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.127+0000 7fce3781b740 1 -- 172.21.15.102:0/1379964610 shutdown_connections 2024-04-24T23:26:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.127+0000 7fce3781b740 1 -- 172.21.15.102:0/1379964610 wait complete. 2024-04-24T23:26:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.130+0000 7fce3781b740 20 client.16377 trim_cache size 0 max 16384 2024-04-24T23:26:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.186+0000 7fce3781b740 1 Processor -- start 2024-04-24T23:26:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.189+0000 7fce3781b740 1 -- start start 2024-04-24T23:26:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.189+0000 7fce3781b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b2431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.189+0000 7fce3781b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b2332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b2431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce3781b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b271b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b2431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50926/0 (socket says 172.21.15.102:50926) 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce2dd9b640 1 -- 172.21.15.102:0/1004069115 learned_addr learned my addr 172.21.15.102:0/1004069115 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce3781b740 1 -- 172.21.15.102:0/1004069115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce301b01f0 con 0x55682b22c420 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce3781b740 1 -- 172.21.15.102:0/1004069115 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fce301adac0 con 0x55682b237410 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce3781b740 1 -- 172.21.15.102:0/1004069115 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ab9c0 con 0x55682b22cd40 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce2e59c640 1 --2- 172.21.15.102:0/1004069115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b2332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce2e59c640 1 -- 172.21.15.102:0/1004069115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 msgr2=0x55682b271b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce2e59c640 1 --2- 172.21.15.102:0/1004069115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b271b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce2e59c640 1 -- 172.21.15.102:0/1004069115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 msgr2=0x55682b2431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce2e59c640 1 --2- 172.21.15.102:0/1004069115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b2431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.190+0000 7fce2e59c640 1 -- 172.21.15.102:0/1004069115 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682b232860 con 0x55682b22cd40 2024-04-24T23:26:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.191+0000 7fce2e59c640 1 --2- 172.21.15.102:0/1004069115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b2332f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55682b2358e0 tx=0x7fce24012fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e579b63c9d736082 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.192+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/1004069115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b2431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.191+0000 7fce17fff640 1 -- 172.21.15.102:0/1004069115 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24017070 con 0x55682b22cd40 2024-04-24T23:26:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.192+0000 7fce17fff640 1 -- 172.21.15.102:0/1004069115 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce2401b7a0 con 0x55682b22cd40 2024-04-24T23:26:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce17fff640 1 -- 172.21.15.102:0/1004069115 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce240138e0 con 0x55682b22cd40 2024-04-24T23:26:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce3781b740 1 -- 172.21.15.102:0/1004069115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 msgr2=0x55682b2332f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce3781b740 1 --2- 172.21.15.102:0/1004069115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b2332f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55682b2358e0 tx=0x7fce24012fa0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce3781b740 1 -- 172.21.15.102:0/1004069115 shutdown_connections 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce3781b740 1 --2- 172.21.15.102:0/1004069115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b271b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce3781b740 1 --2- 172.21.15.102:0/1004069115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b2332f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce3781b740 1 --2- 172.21.15.102:0/1004069115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b2431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce3781b740 1 -- 172.21.15.102:0/1004069115 >> 172.21.15.102:0/1004069115 conn(0x55682b228740 msgr2=0x55682b232be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce3781b740 1 -- 172.21.15.102:0/1004069115 shutdown_connections 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.193+0000 7fce3781b740 1 -- 172.21.15.102:0/1004069115 wait complete. 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.194+0000 7fce3781b740 1 -- start start 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.194+0000 7fce3781b740 1 Processor -- start 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.194+0000 7fce3781b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b23f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.194+0000 7fce3781b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b23fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.194+0000 7fce3781b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b2308d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.194+0000 7fce2e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b23fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.194+0000 7fce2e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b23fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:56480/0 (socket says 172.21.15.102:56480) 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.194+0000 7fce2e59c640 1 -- 172.21.15.102:0/3229594571 learned_addr learned my addr 172.21.15.102:0/3229594571 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.194+0000 7fce3781b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce301abf80 con 0x55682b22c420 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ad3f0 con 0x55682b237410 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ac650 con 0x55682b22cd40 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b2308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce3781b740 10 client.0 _ll_register_callbacks cb 0x7fce24009310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce2e59c640 1 -- 172.21.15.102:0/3229594571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 msgr2=0x55682b2308d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce2e59c640 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b2308d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce2e59c640 1 -- 172.21.15.102:0/3229594571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 msgr2=0x55682b23f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce2e59c640 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b23f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce2e59c640 1 -- 172.21.15.102:0/3229594571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682b231100 con 0x55682b22cd40 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce2e59c640 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b23fc00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fce2401b770 tx=0x7fce2401bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fd8b9b43baf4a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.195+0000 7fce15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:26:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.200+0000 7fce15ffb640 1 -- 172.21.15.102:0/3229594571 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24035060 con 0x55682b22cd40 2024-04-24T23:26:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.200+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce2401b7a0 con 0x55682b22cd40 2024-04-24T23:26:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.200+0000 7fce15ffb640 1 -- 172.21.15.102:0/3229594571 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8623+0+0 (secure 0 0 0) 0x7fce24013880 con 0x55682b22cd40 2024-04-24T23:26:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.200+0000 7fce15ffb640 1 -- 172.21.15.102:0/3229594571 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce24017040 con 0x55682b22cd40 2024-04-24T23:26:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.200+0000 7fce3781b740 10 client.25726 fetch_fsmap learned FSMap version 26 2024-04-24T23:26:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.200+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55682b2f1e50 con 0x55682b22cd40 2024-04-24T23:26:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.202+0000 7fce15ffb640 1 -- 172.21.15.102:0/3229594571 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce2401b7a0 con 0x55682b22cd40 2024-04-24T23:26:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.202+0000 7fce15ffb640 1 -- 172.21.15.102:0/3229594571 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fce24030480 con 0x55682b22cd40 2024-04-24T23:26:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.202+0000 7fce3781b740 10 client.25726 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:26:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.202+0000 7fce3781b740 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:26:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.202+0000 7fce3781b740 20 client.25726 populate_metadata read hostname 'smithi102' 2024-04-24T23:26:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.202+0000 7fce3781b740 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55682b2f2490 0x55682b2f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.202+0000 7fce3781b740 4 client.25726 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:26:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.202+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55682b2239b0 con 0x55682b2f2490 2024-04-24T23:26:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.205+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55682b2f2490 0x55682b2f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.208+0000 7fce2dd9b640 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55682b2f2490 0x55682b2f48b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77f12ac514d18e6 server_cookie=c23818cfd8ecb230 in_seq=0 out_seq=0 2024-04-24T23:26:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.209+0000 7fce15ffb640 10 client.25726 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.251+0000 7fce15ffb640 1 -- 172.21.15.102:0/3229594571 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55682b2239b0 con 0x55682b2f2490 2024-04-24T23:26:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.251+0000 7fce15ffb640 10 client.25726 handle_command_reply: tid=0 2024-04-24T23:26:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.251+0000 7fce3781b740 1 client.25726 shutdown 2024-04-24T23:26:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.258+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 msgr2=0x55682b23fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.258+0000 7fce3781b740 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b23fc00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fce2401b770 tx=0x7fce2401bb60 comp rx=0 tx=0).stop 2024-04-24T23:26:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.258+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 shutdown_connections 2024-04-24T23:26:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.258+0000 7fce3781b740 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55682b237410 0x55682b2308d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.258+0000 7fce3781b740 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55682b22cd40 0x55682b23fc00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.258+0000 7fce3781b740 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55682b2f2490 0x55682b2f48b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.258+0000 7fce3781b740 1 --2- 172.21.15.102:0/3229594571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55682b22c420 0x55682b23f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.258+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 >> 172.21.15.102:0/3229594571 conn(0x55682b228740 msgr2=0x55682b2342b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.259+0000 7fce15ffb640 0 client.25726 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.259+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 shutdown_connections 2024-04-24T23:26:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.261+0000 7fce3781b740 1 -- 172.21.15.102:0/3229594571 wait complete. 2024-04-24T23:26:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.261+0000 7fce3781b740 20 client.25726 trim_cache size 0 max 16384 2024-04-24T23:26:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:26:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:26:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:26:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:26:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.269+0000 7fce2f59e640 1 -- 172.21.15.102:0/2909159156 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fce180a2df0 msgr2=0x7fce180a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.269+0000 7fce2f59e640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fce180a2df0 0x7fce180a52b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce2003bfd0 tx=0x7fce2002a4d0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.269+0000 7fce2f59e640 1 -- 172.21.15.102:0/2909159156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce301101e0 msgr2=0x7fce30115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.269+0000 7fce2f59e640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce301101e0 0x7fce30115b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fce2002a9f0 tx=0x7fce20003cd0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.273+0000 7fce2f59e640 1 -- 172.21.15.102:0/2909159156 shutdown_connections 2024-04-24T23:26:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.273+0000 7fce2f59e640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce301101e0 0x7fce30115b80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.274+0000 7fce2f59e640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3010fe00 0x7fce30115640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.274+0000 7fce2f59e640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fce180a2df0 0x7fce180a52b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.274+0000 7fce2f59e640 1 --2- 172.21.15.102:0/2909159156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce3010e410 0x7fce3011c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.274+0000 7fce2f59e640 1 -- 172.21.15.102:0/2909159156 >> 172.21.15.102:0/2909159156 conn(0x7fce3006c680 msgr2=0x7fce30070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.275+0000 7fce2f59e640 1 -- 172.21.15.102:0/2909159156 shutdown_connections 2024-04-24T23:26:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:18.275+0000 7fce2f59e640 1 -- 172.21.15.102:0/2909159156 wait complete. 2024-04-24T23:26:19.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:19 smithi183 ceph-mon[33137]: pgmap v414: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2024-04-24T23:26:19.822 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:19 smithi102 ceph-mon[29216]: pgmap v414: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2024-04-24T23:26:19.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:19 smithi161 ceph-mon[32709]: pgmap v414: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2024-04-24T23:26:21.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:21 smithi183 ceph-mon[33137]: pgmap v415: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 22 MiB/s wr, 5.45k op/s 2024-04-24T23:26:21.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:21 smithi102 ceph-mon[29216]: pgmap v415: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 22 MiB/s wr, 5.45k op/s 2024-04-24T23:26:21.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:21 smithi161 ceph-mon[32709]: pgmap v415: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 22 MiB/s wr, 5.45k op/s 2024-04-24T23:26:23.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:23 smithi183 ceph-mon[33137]: pgmap v416: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2024-04-24T23:26:23.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:23 smithi102 ceph-mon[29216]: pgmap v416: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2024-04-24T23:26:23.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:23 smithi161 ceph-mon[32709]: pgmap v416: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2024-04-24T23:26:25.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:25 smithi102 ceph-mon[29216]: pgmap v417: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 22 MiB/s wr, 5.67k op/s 2024-04-24T23:26:25.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:25 smithi161 ceph-mon[32709]: pgmap v417: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 22 MiB/s wr, 5.67k op/s 2024-04-24T23:26:26.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:25 smithi183 ceph-mon[33137]: pgmap v417: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 22 MiB/s wr, 5.67k op/s 2024-04-24T23:26:27.825 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:27 smithi102 ceph-mon[29216]: pgmap v418: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 20 MiB/s wr, 4.93k op/s 2024-04-24T23:26:27.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:27 smithi161 ceph-mon[32709]: pgmap v418: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 20 MiB/s wr, 4.93k op/s 2024-04-24T23:26:28.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:27 smithi183 ceph-mon[33137]: pgmap v418: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 20 MiB/s wr, 4.93k op/s 2024-04-24T23:26:29.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:29 smithi102 ceph-mon[29216]: pgmap v419: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2024-04-24T23:26:29.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:29 smithi161 ceph-mon[32709]: pgmap v419: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2024-04-24T23:26:30.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:29 smithi183 ceph-mon[33137]: pgmap v419: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2024-04-24T23:26:31.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:31 smithi102 ceph-mon[29216]: pgmap v420: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 23 MiB/s wr, 5.87k op/s 2024-04-24T23:26:31.323 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:31 smithi102 ceph-mon[29216]: osdmap e105: 12 total, 12 up, 12 in 2024-04-24T23:26:31.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:31 smithi161 ceph-mon[32709]: pgmap v420: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 23 MiB/s wr, 5.87k op/s 2024-04-24T23:26:31.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:31 smithi161 ceph-mon[32709]: osdmap e105: 12 total, 12 up, 12 in 2024-04-24T23:26:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.498+0000 7f340fd53640 1 -- 172.21.15.102:0/3711586803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3408105940 msgr2=0x7f3408105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.498+0000 7f340fd53640 1 --2- 172.21.15.102:0/3711586803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3408105940 0x7f3408105dd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009900 tx=0x7f33f802a4b0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.498+0000 7f340fd53640 1 -- 172.21.15.102:0/3711586803 shutdown_connections 2024-04-24T23:26:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.498+0000 7f340fd53640 1 --2- 172.21.15.102:0/3711586803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3408105940 0x7f3408105dd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.498+0000 7f340fd53640 1 --2- 172.21.15.102:0/3711586803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3408100e60 0x7f3408105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.498+0000 7f340fd53640 1 --2- 172.21.15.102:0/3711586803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408100540 0x7f3408100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.498+0000 7f340fd53640 1 -- 172.21.15.102:0/3711586803 >> 172.21.15.102:0/3711586803 conn(0x7f34080fbfb0 msgr2=0x7f34080fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.499+0000 7f340fd53640 1 -- 172.21.15.102:0/3711586803 shutdown_connections 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.499+0000 7f340fd53640 1 -- 172.21.15.102:0/3711586803 wait complete. 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.499+0000 7f340fd53640 1 Processor -- start 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.499+0000 7f340fd53640 1 -- start start 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340fd53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408100540 0x7f3408072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340fd53640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3408100e60 0x7f340806bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340daef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408100540 0x7f3408072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340fd53640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3408105940 0x7f340806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340daef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408100540 0x7f3408072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36164/0 (socket says 172.21.15.102:36164) 2024-04-24T23:26:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340daef640 1 -- 172.21.15.102:0/2390636851 learned_addr learned my addr 172.21.15.102:0/2390636851 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3408117e30 con 0x7f3408100e60 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3408117cb0 con 0x7f3408105940 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3408117fb0 con 0x7f3408100540 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.500+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3408100e60 0x7f340806bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.501+0000 7f340daef640 1 -- 172.21.15.102:0/2390636851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3408105940 msgr2=0x7f340806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.501+0000 7f340daef640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3408105940 0x7f340806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.501+0000 7f340daef640 1 -- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3408100e60 msgr2=0x7f340806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.501+0000 7f340daef640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3408100e60 0x7f340806bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.501+0000 7f340daef640 1 -- 172.21.15.102:0/2390636851 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340806cab0 con 0x7f3408100540 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.501+0000 7f340daef640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408100540 0x7f3408072f30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f340400d950 tx=0x7f340400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.502+0000 7f33feffd640 1 -- 172.21.15.102:0/2390636851 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404017070 con 0x7f3408100540 2024-04-24T23:26:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.502+0000 7f33feffd640 1 -- 172.21.15.102:0/2390636851 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3404009c30 con 0x7f3408100540 2024-04-24T23:26:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.504+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340806cda0 con 0x7f3408100540 2024-04-24T23:26:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.504+0000 7f33feffd640 1 -- 172.21.15.102:0/2390636851 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340400f3c0 con 0x7f3408100540 2024-04-24T23:26:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.504+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3408070d80 con 0x7f3408100540 2024-04-24T23:26:31.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:31 smithi183 ceph-mon[33137]: pgmap v420: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 23 MiB/s wr, 5.87k op/s 2024-04-24T23:26:31.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:31 smithi183 ceph-mon[33137]: osdmap e105: 12 total, 12 up, 12 in 2024-04-24T23:26:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.508+0000 7f3412a5e740 1 Processor -- start 2024-04-24T23:26:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f3412a5e740 1 -- start start 2024-04-24T23:26:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c3c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 0x55bab7c57e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c583d0 0x55bab7c5a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f3412a5e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bab7ba0cb0 con 0x55bab7c3cbd0 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f3412a5e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bab7b495b0 con 0x55bab7c3c2b0 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f3412a5e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bab7b9cc70 con 0x55bab7c583d0 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f340daef640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 0x55bab7c57e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f340daef640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 0x55bab7c57e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49078/0 (socket says 172.21.15.102:49078) 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f340daef640 1 -- 172.21.15.102:0/2670065774 learned_addr learned my addr 172.21.15.102:0/2670065774 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f340e2f0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c3c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.510+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2670065774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c583d0 0x55bab7c5a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.511+0000 7f340d2ee640 1 -- 172.21.15.102:0/2670065774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 msgr2=0x55bab7c3c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.511+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2670065774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c3c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.511+0000 7f340d2ee640 1 -- 172.21.15.102:0/2670065774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 msgr2=0x55bab7c57e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.511+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2670065774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 0x55bab7c57e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.511+0000 7f340d2ee640 1 -- 172.21.15.102:0/2670065774 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bab78b5210 con 0x55bab7c583d0 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.511+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2670065774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c583d0 0x55bab7c5a7c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f340806d800 tx=0x7f3400010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0a2805cb6b5749 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.512+0000 7f33f3fff640 1 -- 172.21.15.102:0/2670065774 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400016070 con 0x55bab7c583d0 2024-04-24T23:26:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.512+0000 7f33f3fff640 1 -- 172.21.15.102:0/2670065774 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f340001bb10 con 0x55bab7c583d0 2024-04-24T23:26:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.511+0000 7f33feffd640 1 -- 172.21.15.102:0/2390636851 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3404010460 con 0x7f3408100540 2024-04-24T23:26:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.513+0000 7f3412a5e740 1 -- 172.21.15.102:0/2670065774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c583d0 msgr2=0x55bab7c5a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.513+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2670065774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c583d0 0x55bab7c5a7c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f340806d800 tx=0x7f3400010740 comp rx=0 tx=0).stop 2024-04-24T23:26:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.513+0000 7f3412a5e740 1 -- 172.21.15.102:0/2670065774 shutdown_connections 2024-04-24T23:26:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.513+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2670065774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c583d0 0x55bab7c5a7c0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f340806d800 tx=0x7f3400010740 comp rx=0 tx=0).stop 2024-04-24T23:26:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.513+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2670065774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 0x55bab7c57e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.513+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2670065774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c3c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.513+0000 7f3412a5e740 1 -- 172.21.15.102:0/2670065774 >> 172.21.15.102:0/2670065774 conn(0x55bab7c386f0 msgr2=0x55bab7c38ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.519+0000 7f33feffd640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f33f40a2d60 0x7f33f40a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.519+0000 7f33feffd640 1 -- 172.21.15.102:0/2390636851 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7f34040ce9e0 con 0x7f3408100540 2024-04-24T23:26:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.519+0000 7f3412a5e740 1 -- 172.21.15.102:0/2670065774 shutdown_connections 2024-04-24T23:26:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.519+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f33f40a2d60 0x7f33f40a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f3412a5e740 1 -- 172.21.15.102:0/2670065774 wait complete. 2024-04-24T23:26:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f3412a5e740 1 -- start start 2024-04-24T23:26:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f3412a5e740 1 Processor -- start 2024-04-24T23:26:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f33f40a2d60 0x7f33f40a5220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3400006010 tx=0x7f3400010ee0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3c2b0 0x55bab7c49920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3cbd0 0x55bab7c49e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c45100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f3412a5e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bab7c3f0b0 con 0x55bab7c3c2b0 2024-04-24T23:26:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f3412a5e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bab7b5cb50 con 0x55bab7c3cbd0 2024-04-24T23:26:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.520+0000 7f3412a5e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bab7c3f230 con 0x55bab7c44c80 2024-04-24T23:26:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f3412a5e740 10 client.0 _ll_register_callbacks cb 0x55bab7b97ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340daef640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3c2b0 0x55bab7c49920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340daef640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3c2b0 0x55bab7c49920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49086/0 (socket says 172.21.15.102:49086) 2024-04-24T23:26:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340daef640 1 -- 172.21.15.102:0/2959096977 learned_addr learned my addr 172.21.15.102:0/2959096977 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340e2f0640 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c45100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3cbd0 0x55bab7c49e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340d2ee640 1 -- 172.21.15.102:0/2959096977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 msgr2=0x55bab7c45100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c45100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340d2ee640 1 -- 172.21.15.102:0/2959096977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3c2b0 msgr2=0x55bab7c49920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3c2b0 0x55bab7c49920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.521+0000 7f340d2ee640 1 -- 172.21.15.102:0/2959096977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bab7981620 con 0x55bab7c3cbd0 2024-04-24T23:26:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.522+0000 7f340d2ee640 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3cbd0 0x55bab7c49e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55bab7c3dc10 tx=0x7f3400009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=247bf918fc4e74f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.522+0000 7f33f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-24T23:26:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.525+0000 7f33f17fa640 1 -- 172.21.15.102:0/2959096977 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340002d070 con 0x55bab7c3cbd0 2024-04-24T23:26:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.525+0000 7f3412a5e740 1 -- 172.21.15.102:0/2959096977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bab7bae1f0 con 0x55bab7c3cbd0 2024-04-24T23:26:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.525+0000 7f33f17fa640 1 -- 172.21.15.102:0/2959096977 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7f3400005790 con 0x55bab7c3cbd0 2024-04-24T23:26:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.526+0000 7f33f17fa640 1 -- 172.21.15.102:0/2959096977 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3400039050 con 0x55bab7c3cbd0 2024-04-24T23:26:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.527+0000 7f3412a5e740 10 client.25820 fetch_fsmap learned FSMap version 26 2024-04-24T23:26:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.527+0000 7f3412a5e740 1 -- 172.21.15.102:0/2959096977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bab7c54200 con 0x55bab7c3cbd0 2024-04-24T23:26:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.527+0000 7f33f17fa640 1 -- 172.21.15.102:0/2959096977 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bab7bae1f0 con 0x55bab7c3cbd0 2024-04-24T23:26:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.528+0000 7f33f17fa640 1 -- 172.21.15.102:0/2959096977 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f340000f040 con 0x55bab7c3cbd0 2024-04-24T23:26:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.528+0000 7f3412a5e740 10 client.25820 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:26:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.528+0000 7f3412a5e740 10 client.25820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:26:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.528+0000 7f3412a5e740 20 client.25820 populate_metadata read hostname 'smithi102' 2024-04-24T23:26:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.528+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bab7c558c0 0x55bab7c57d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.528+0000 7f3412a5e740 4 client.25820 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.528+0000 7f3412a5e740 1 -- 172.21.15.102:0/2959096977 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bab7ba8540 con 0x55bab7c558c0 2024-04-24T23:26:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.530+0000 7f340daef640 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bab7c558c0 0x55bab7c57d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.539+0000 7f340daef640 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bab7c558c0 0x55bab7c57d80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e7677b0cd89004 server_cookie=d4e4173241d0f6f6 in_seq=0 out_seq=0 2024-04-24T23:26:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.542+0000 7f33f17fa640 10 client.25820 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.545+0000 7f33f17fa640 1 -- 172.21.15.102:0/2959096977 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bab7ba8540 con 0x55bab7c558c0 2024-04-24T23:26:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.545+0000 7f33f17fa640 10 client.25820 handle_command_reply: tid=0 2024-04-24T23:26:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.547+0000 7f3412a5e740 1 client.25820 shutdown 2024-04-24T23:26:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.555+0000 7f3412a5e740 1 -- 172.21.15.102:0/2959096977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3cbd0 msgr2=0x55bab7c49e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.555+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3cbd0 0x55bab7c49e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55bab7c3dc10 tx=0x7f3400009750 comp rx=0 tx=0).stop 2024-04-24T23:26:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.560+0000 7f3412a5e740 1 -- 172.21.15.102:0/2959096977 shutdown_connections 2024-04-24T23:26:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.560+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c45100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.560+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3cbd0 0x55bab7c49e60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.560+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bab7c558c0 0x55bab7c57d80 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.560+0000 7f3412a5e740 1 --2- 172.21.15.102:0/2959096977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3c2b0 0x55bab7c49920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.560+0000 7f3412a5e740 1 -- 172.21.15.102:0/2959096977 >> 172.21.15.102:0/2959096977 conn(0x55bab7c386f0 msgr2=0x55bab7c3e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.562+0000 7f33f17fa640 0 client.25820 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.565+0000 7f3412a5e740 1 -- 172.21.15.102:0/2959096977 shutdown_connections 2024-04-24T23:26:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.565+0000 7f3412a5e740 1 -- 172.21.15.102:0/2959096977 wait complete. 2024-04-24T23:26:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.570+0000 7f3412a5e740 20 client.25820 trim_cache size 0 max 16384 2024-04-24T23:26:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.620+0000 7f3412a5e740 1 Processor -- start 2024-04-24T23:26:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.628+0000 7f3412a5e740 1 -- start start 2024-04-24T23:26:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.628+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c60290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.628+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c3cbd0 0x55bab7c607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.628+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c44c80 0x55bab7c3ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.628+0000 7f3412a5e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34081a4db0 con 0x55bab7c44c80 2024-04-24T23:26:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.628+0000 7f3412a5e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f340810f600 con 0x55bab7c3c2b0 2024-04-24T23:26:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.628+0000 7f3412a5e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f340810d500 con 0x55bab7c3cbd0 2024-04-24T23:26:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.630+0000 7f340daef640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c60290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.630+0000 7f340d2ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c3cbd0 0x55bab7c607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.630+0000 7f340daef640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c60290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:59610/0 (socket says 172.21.15.102:59610) 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.630+0000 7f340daef640 1 -- 172.21.15.102:0/1481790645 learned_addr learned my addr 172.21.15.102:0/1481790645 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.631+0000 7f340daef640 1 -- 172.21.15.102:0/1481790645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c3cbd0 msgr2=0x55bab7c607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.631+0000 7f340daef640 1 --2- 172.21.15.102:0/1481790645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c3cbd0 0x55bab7c607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.631+0000 7f340daef640 1 -- 172.21.15.102:0/1481790645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c44c80 msgr2=0x55bab7c3ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.631+0000 7f340daef640 1 --2- 172.21.15.102:0/1481790645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c44c80 0x55bab7c3ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.631+0000 7f340daef640 1 -- 172.21.15.102:0/1481790645 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bab7c431d0 con 0x55bab7c3c2b0 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.632+0000 7f340daef640 1 --2- 172.21.15.102:0/1481790645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c60290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f34040978b0 tx=0x7f34040d0e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=305ddca183092018 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.633+0000 7f33f37fe640 1 -- 172.21.15.102:0/1481790645 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340409b5e0 con 0x55bab7c3c2b0 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.633+0000 7f33f37fe640 1 -- 172.21.15.102:0/1481790645 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3404009de0 con 0x55bab7c3c2b0 2024-04-24T23:26:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.633+0000 7f33f37fe640 1 -- 172.21.15.102:0/1481790645 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404010750 con 0x55bab7c3c2b0 2024-04-24T23:26:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.634+0000 7f3412a5e740 1 -- 172.21.15.102:0/1481790645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 msgr2=0x55bab7c60290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.634+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1481790645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c60290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f34040978b0 tx=0x7f34040d0e70 comp rx=0 tx=0).stop 2024-04-24T23:26:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.640+0000 7f3412a5e740 1 -- 172.21.15.102:0/1481790645 shutdown_connections 2024-04-24T23:26:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.640+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1481790645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c44c80 0x55bab7c3ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.640+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1481790645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c3cbd0 0x55bab7c607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.640+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1481790645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c60290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.640+0000 7f3412a5e740 1 -- 172.21.15.102:0/1481790645 >> 172.21.15.102:0/1481790645 conn(0x55bab7c386f0 msgr2=0x55bab7c38ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.640+0000 7f3412a5e740 1 -- 172.21.15.102:0/1481790645 shutdown_connections 2024-04-24T23:26:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.640+0000 7f3412a5e740 1 -- 172.21.15.102:0/1481790645 wait complete. 2024-04-24T23:26:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.640+0000 7f3412a5e740 1 -- start start 2024-04-24T23:26:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.640+0000 7f3412a5e740 1 Processor -- start 2024-04-24T23:26:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.641+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c4c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.641+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 0x55bab7c4ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.641+0000 7f3412a5e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c4cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.641+0000 7f3412a5e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f340810dac0 con 0x55bab7c3cbd0 2024-04-24T23:26:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.641+0000 7f3412a5e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f340810ef30 con 0x55bab7c3c2b0 2024-04-24T23:26:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.641+0000 7f3412a5e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f340810e190 con 0x55bab7c44c80 2024-04-24T23:26:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.641+0000 7f3412a5e740 10 client.0 _ll_register_callbacks cb 0x7f34040f6410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.643+0000 7f340e2f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c4cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.643+0000 7f340e2f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c4cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:36204/0 (socket says 172.21.15.102:36204) 2024-04-24T23:26:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.643+0000 7f340e2f0640 1 -- 172.21.15.102:0/1017896804 learned_addr learned my addr 172.21.15.102:0/1017896804 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.644+0000 7f340daef640 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c4c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.645+0000 7f340e2f0640 1 -- 172.21.15.102:0/1017896804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 msgr2=0x55bab7c4c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.645+0000 7f340e2f0640 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c4c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.645+0000 7f340e2f0640 1 -- 172.21.15.102:0/1017896804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 msgr2=0x55bab7c4ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.645+0000 7f340e2f0640 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 0x55bab7c4ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.645+0000 7f340e2f0640 1 -- 172.21.15.102:0/1017896804 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bab7cbcbf0 con 0x55bab7c44c80 2024-04-24T23:26:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.649+0000 7f340e2f0640 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c4cfc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33f8033000 tx=0x7f33f8004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906258110f80878f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.655+0000 7f33f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-24T23:26:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.655+0000 7f33f17fa640 1 -- 172.21.15.102:0/1017896804 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8035040 con 0x55bab7c44c80 2024-04-24T23:26:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.655+0000 7f33f17fa640 1 -- 172.21.15.102:0/1017896804 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7f33f8005450 con 0x55bab7c44c80 2024-04-24T23:26:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.656+0000 7f3412a5e740 1 -- 172.21.15.102:0/1017896804 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3404009de0 con 0x55bab7c44c80 2024-04-24T23:26:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.657+0000 7f33f17fa640 1 -- 172.21.15.102:0/1017896804 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f33f80046c0 con 0x55bab7c44c80 2024-04-24T23:26:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.662+0000 7f3412a5e740 10 client.25744 fetch_fsmap learned FSMap version 26 2024-04-24T23:26:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.662+0000 7f3412a5e740 1 -- 172.21.15.102:0/1017896804 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bab7cbdee0 con 0x55bab7c44c80 2024-04-24T23:26:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.664+0000 7f33f17fa640 1 -- 172.21.15.102:0/1017896804 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3404009de0 con 0x55bab7c44c80 2024-04-24T23:26:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.664+0000 7f33f17fa640 1 -- 172.21.15.102:0/1017896804 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f33f8003af0 con 0x55bab7c44c80 2024-04-24T23:26:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.667+0000 7f3412a5e740 10 client.25744 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:26:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.667+0000 7f3412a5e740 10 client.25744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:26:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.667+0000 7f3412a5e740 20 client.25744 populate_metadata read hostname 'smithi102' 2024-04-24T23:26:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.667+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bab7cbe560 0x55bab7cc0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.667+0000 7f3412a5e740 4 client.25744 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:26:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.667+0000 7f3412a5e740 1 -- 172.21.15.102:0/1017896804 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bab7b268e0 con 0x55bab7cbe560 2024-04-24T23:26:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.670+0000 7f340daef640 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bab7cbe560 0x55bab7cc0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.670+0000 7f340daef640 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bab7cbe560 0x55bab7cc0980 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc8280b385988110 server_cookie=7e7a070780e0a8a5 in_seq=0 out_seq=0 2024-04-24T23:26:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.670+0000 7f33f17fa640 10 client.25744 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.893+0000 7f33f17fa640 1 -- 172.21.15.102:0/1017896804 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bab7b268e0 con 0x55bab7cbe560 2024-04-24T23:26:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.893+0000 7f33f17fa640 10 client.25744 handle_command_reply: tid=0 2024-04-24T23:26:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.894+0000 7f3412a5e740 1 client.25744 shutdown 2024-04-24T23:26:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.898+0000 7f3412a5e740 1 -- 172.21.15.102:0/1017896804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 msgr2=0x55bab7c4cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.898+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c4cfc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33f8033000 tx=0x7f33f8004480 comp rx=0 tx=0).stop 2024-04-24T23:26:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.898+0000 7f3412a5e740 1 -- 172.21.15.102:0/1017896804 shutdown_connections 2024-04-24T23:26:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.898+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bab7c44c80 0x55bab7c4cfc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.902 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.898+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x55bab7cbe560 0x55bab7cc0980 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.898+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bab7c3cbd0 0x55bab7c4ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.898+0000 7f3412a5e740 1 --2- 172.21.15.102:0/1017896804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bab7c3c2b0 0x55bab7c4c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.898+0000 7f3412a5e740 1 -- 172.21.15.102:0/1017896804 >> 172.21.15.102:0/1017896804 conn(0x55bab7c386f0 msgr2=0x55bab7c3ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.900+0000 7f33f17fa640 0 client.25744 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.900+0000 7f3412a5e740 1 -- 172.21.15.102:0/1017896804 shutdown_connections 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.900+0000 7f3412a5e740 1 -- 172.21.15.102:0/1017896804 wait complete. 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.900+0000 7f3412a5e740 20 client.25744 trim_cache size 0 max 16384 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:26:31.903 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:26:31.904 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.903+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f33f40a2d60 msgr2=0x7f33f40a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:31.904 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.903+0000 7f340fd53640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f33f40a2d60 0x7f33f40a5220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3400006010 tx=0x7f3400010ee0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.904+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408100540 msgr2=0x7f3408072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:31.905 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.904+0000 7f340fd53640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408100540 0x7f3408072f30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f340400d950 tx=0x7f340400de10 comp rx=0 tx=0).stop 2024-04-24T23:26:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.904+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 shutdown_connections 2024-04-24T23:26:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.904+0000 7f340fd53640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3408105940 0x7f340806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.904+0000 7f340fd53640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f33f40a2d60 0x7f33f40a5220 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.904+0000 7f340fd53640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3408100e60 0x7f340806bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.906 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.904+0000 7f340fd53640 1 --2- 172.21.15.102:0/2390636851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3408100540 0x7f3408072f30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.904+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 >> 172.21.15.102:0/2390636851 conn(0x7f34080fbfb0 msgr2=0x7f34080fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.905+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 shutdown_connections 2024-04-24T23:26:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:31.905+0000 7f340fd53640 1 -- 172.21.15.102:0/2390636851 wait complete. 2024-04-24T23:26:33.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:33 smithi102 ceph-mon[29216]: pgmap v422: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2024-04-24T23:26:33.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:33 smithi161 ceph-mon[32709]: pgmap v422: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2024-04-24T23:26:33.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:33 smithi183 ceph-mon[33137]: pgmap v422: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2024-04-24T23:26:35.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:35 smithi161 ceph-mon[32709]: pgmap v423: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 21 MiB/s wr, 5.21k op/s 2024-04-24T23:26:35.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:35 smithi183 ceph-mon[33137]: pgmap v423: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 21 MiB/s wr, 5.21k op/s 2024-04-24T23:26:35.574 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:35 smithi102 ceph-mon[29216]: pgmap v423: 97 pgs: 97 active+clean; 19 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 21 MiB/s wr, 5.21k op/s 2024-04-24T23:26:36.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:36 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:36.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:36 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:36.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:36 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:37.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:37 smithi161 ceph-mon[32709]: pgmap v424: 97 pgs: 97 active+clean; 19 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2024-04-24T23:26:37.473 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:37 smithi102 ceph-mon[29216]: pgmap v424: 97 pgs: 97 active+clean; 19 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2024-04-24T23:26:37.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:37 smithi183 ceph-mon[33137]: pgmap v424: 97 pgs: 97 active+clean; 19 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2024-04-24T23:26:39.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:39 smithi161 ceph-mon[32709]: pgmap v425: 97 pgs: 97 active+clean; 19 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2024-04-24T23:26:39.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:39 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:39.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:39 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:39.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:39 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:26:39.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:39 smithi183 ceph-mon[33137]: pgmap v425: 97 pgs: 97 active+clean; 19 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2024-04-24T23:26:39.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:39 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:39.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:39 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:39.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:39 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:26:39.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:39 smithi102 ceph-mon[29216]: pgmap v425: 97 pgs: 97 active+clean; 19 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2024-04-24T23:26:39.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:39 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:39.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:39 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:39.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:39 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:26:41.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:41 smithi161 ceph-mon[32709]: pgmap v426: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2024-04-24T23:26:41.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:41 smithi183 ceph-mon[33137]: pgmap v426: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2024-04-24T23:26:41.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:41 smithi102 ceph-mon[29216]: pgmap v426: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2024-04-24T23:26:43.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:43 smithi161 ceph-mon[32709]: pgmap v427: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2024-04-24T23:26:43.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:43 smithi183 ceph-mon[33137]: pgmap v427: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2024-04-24T23:26:43.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:43 smithi102 ceph-mon[29216]: pgmap v427: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2024-04-24T23:26:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.145+0000 7f11da58c640 1 -- 172.21.15.102:0/1471233851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d40ff7a0 msgr2=0x7f11d40ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.145+0000 7f11da58c640 1 --2- 172.21.15.102:0/1471233851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d40ff7a0 0x7f11d40ffc20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11cc005fd0 tx=0x7f11cc02b3f0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.149+0000 7f11da58c640 1 -- 172.21.15.102:0/1471233851 shutdown_connections 2024-04-24T23:26:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.149+0000 7f11da58c640 1 --2- 172.21.15.102:0/1471233851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11d4100160 0x7f11d410c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.149+0000 7f11da58c640 1 --2- 172.21.15.102:0/1471233851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d40ff7a0 0x7f11d40ffc20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.149+0000 7f11da58c640 1 --2- 172.21.15.102:0/1471233851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d41057b0 0x7f11d4105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.149+0000 7f11da58c640 1 -- 172.21.15.102:0/1471233851 >> 172.21.15.102:0/1471233851 conn(0x7f11d40765a0 msgr2=0x7f11d40fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.153+0000 7f11da58c640 1 -- 172.21.15.102:0/1471233851 shutdown_connections 2024-04-24T23:26:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.157+0000 7f11da58c640 1 -- 172.21.15.102:0/1471233851 wait complete. 2024-04-24T23:26:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.157+0000 7f11da58c640 1 Processor -- start 2024-04-24T23:26:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.158+0000 7f11da58c640 1 -- start start 2024-04-24T23:26:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.158+0000 7f11da58c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d40ff7a0 0x7f11d410c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.158+0000 7f11da58c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d4100160 0x7f11d4109620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.158+0000 7f11da58c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11d41057b0 0x7f11d4109bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.158+0000 7f11da58c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410f3b0 con 0x7f11d41057b0 2024-04-24T23:26:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.158+0000 7f11da58c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410f230 con 0x7f11d40ff7a0 2024-04-24T23:26:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.158+0000 7f11da58c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410f530 con 0x7f11d4100160 2024-04-24T23:26:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.162+0000 7f11d8d89640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d4100160 0x7f11d4109620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.162+0000 7f11d958a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d40ff7a0 0x7f11d410c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.162+0000 7f11d9d8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11d41057b0 0x7f11d4109bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.162+0000 7f11d9d8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11d41057b0 0x7f11d4109bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37866/0 (socket says 172.21.15.102:37866) 2024-04-24T23:26:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.162+0000 7f11d9d8b640 1 -- 172.21.15.102:0/924749408 learned_addr learned my addr 172.21.15.102:0/924749408 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.170+0000 7f11d8d89640 1 -- 172.21.15.102:0/924749408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d40ff7a0 msgr2=0x7f11d410c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.170+0000 7f11d8d89640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d40ff7a0 0x7f11d410c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.170+0000 7f11d8d89640 1 -- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11d41057b0 msgr2=0x7f11d4109bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.170+0000 7f11d8d89640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11d41057b0 0x7f11d4109bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.170+0000 7f11d8d89640 1 -- 172.21.15.102:0/924749408 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d40721b0 con 0x7f11d4100160 2024-04-24T23:26:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.171+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11d41057b0 0x7f11d4109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:26:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.173+0000 7f11d8d89640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d4100160 0x7f11d4109620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11cc035040 tx=0x7f11cc00a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.175+0000 7f11ca7fc640 1 -- 172.21.15.102:0/924749408 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc037040 con 0x7f11d4100160 2024-04-24T23:26:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.175+0000 7f11ca7fc640 1 -- 172.21.15.102:0/924749408 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11cc003e20 con 0x7f11d4100160 2024-04-24T23:26:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.175+0000 7f11ca7fc640 1 -- 172.21.15.102:0/924749408 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc037040 con 0x7f11d4100160 2024-04-24T23:26:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.177+0000 7f11da58c640 1 -- 172.21.15.102:0/924749408 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d4072440 con 0x7f11d4100160 2024-04-24T23:26:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.177+0000 7f11da58c640 1 -- 172.21.15.102:0/924749408 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d4072940 con 0x7f11d4100160 2024-04-24T23:26:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.181+0000 7f11de5db740 1 Processor -- start 2024-04-24T23:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.181+0000 7f11de5db740 1 -- start start 2024-04-24T23:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.182+0000 7f11de5db740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533f420 0x555a6533f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.182+0000 7f11de5db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533fd40 0x555a65349ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.182+0000 7f11de5db740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6534a410 0x555a6534c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.182+0000 7f11de5db740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555a652a3cb0 con 0x555a6534a410 2024-04-24T23:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.182+0000 7f11de5db740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555a6524c5b0 con 0x555a6533f420 2024-04-24T23:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.182+0000 7f11de5db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a6529fc70 con 0x555a6533fd40 2024-04-24T23:26:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.183+0000 7f11d9d8b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533f420 0x555a6533f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.184+0000 7f11d9d8b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533f420 0x555a6533f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:38724/0 (socket says 172.21.15.102:38724) 2024-04-24T23:26:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.184+0000 7f11d9d8b640 1 -- 172.21.15.102:0/3177115407 learned_addr learned my addr 172.21.15.102:0/3177115407 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.184+0000 7f11d958a640 1 --2- 172.21.15.102:0/3177115407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533fd40 0x555a65349ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.188+0000 7f11ca7fc640 1 -- 172.21.15.102:0/924749408 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f11cc003970 con 0x7f11d4100160 2024-04-24T23:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.189+0000 7f11ca7fc640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11c00a3130 0x7f11c00a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.189+0000 7f11d9d8b640 1 -- 172.21.15.102:0/3177115407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533fd40 msgr2=0x555a65349ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.189+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/3177115407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533fd40 0x555a65349ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.189+0000 7f11d9d8b640 1 -- 172.21.15.102:0/3177115407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6534a410 msgr2=0x555a6534c800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.189+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/3177115407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6534a410 0x555a6534c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.189+0000 7f11d9d8b640 1 -- 172.21.15.102:0/3177115407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a64fb8210 con 0x555a6533f420 2024-04-24T23:26:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.189+0000 7f11ca7fc640 1 -- 172.21.15.102:0/924749408 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7f11cc007a50 con 0x7f11d4100160 2024-04-24T23:26:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.189+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/3177115407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533f420 0x555a6533f800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f11c4010550 tx=0x7f11c400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f171251fb0b7a366 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.194+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11c00a3130 0x7f11c00a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.195+0000 7f11bf7fe640 1 -- 172.21.15.102:0/3177115407 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c40118c0 con 0x555a6533f420 2024-04-24T23:26:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.195+0000 7f11bf7fe640 1 -- 172.21.15.102:0/3177115407 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11c401c480 con 0x555a6533f420 2024-04-24T23:26:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.195+0000 7f11bf7fe640 1 -- 172.21.15.102:0/3177115407 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c400e6b0 con 0x555a6533f420 2024-04-24T23:26:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.196+0000 7f11de5db740 1 -- 172.21.15.102:0/3177115407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533f420 msgr2=0x555a6533f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.196+0000 7f11de5db740 1 --2- 172.21.15.102:0/3177115407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533f420 0x555a6533f800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f11c4010550 tx=0x7f11c400bf90 comp rx=0 tx=0).stop 2024-04-24T23:26:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.199+0000 7f11de5db740 1 -- 172.21.15.102:0/3177115407 shutdown_connections 2024-04-24T23:26:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.199+0000 7f11de5db740 1 --2- 172.21.15.102:0/3177115407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6534a410 0x555a6534c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.199+0000 7f11de5db740 1 --2- 172.21.15.102:0/3177115407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533fd40 0x555a65349ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.199+0000 7f11de5db740 1 --2- 172.21.15.102:0/3177115407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533f420 0x555a6533f800 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.199+0000 7f11de5db740 1 -- 172.21.15.102:0/3177115407 >> 172.21.15.102:0/3177115407 conn(0x555a6533b740 msgr2=0x555a6533bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.206+0000 7f11de5db740 1 -- 172.21.15.102:0/3177115407 shutdown_connections 2024-04-24T23:26:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.208+0000 7f11de5db740 1 -- 172.21.15.102:0/3177115407 wait complete. 2024-04-24T23:26:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.209+0000 7f11de5db740 1 -- start start 2024-04-24T23:26:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.209+0000 7f11de5db740 1 Processor -- start 2024-04-24T23:26:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.212+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11c00a3130 0x7f11c00a55f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11c4010550 tx=0x7f11c400bf90 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.213+0000 7f11de5db740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 0x555a65347390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.213+0000 7f11de5db740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a653478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.213+0000 7f11de5db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a65347e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.213+0000 7f11de5db740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555a6534d200 con 0x555a6533f420 2024-04-24T23:26:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.213+0000 7f11de5db740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555a6525fb50 con 0x555a6533fd40 2024-04-24T23:26:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.213+0000 7f11de5db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a6534d380 con 0x555a6534a410 2024-04-24T23:26:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.213+0000 7f11de5db740 10 client.0 _ll_register_callbacks cb 0x555a6529aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.214+0000 7f11d9d8b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a65347e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.214+0000 7f11d9d8b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a65347e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:41776/0 (socket says 172.21.15.102:41776) 2024-04-24T23:26:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.214+0000 7f11d9d8b640 1 -- 172.21.15.102:0/2627246944 learned_addr learned my addr 172.21.15.102:0/2627246944 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.214+0000 7f11d958a640 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 0x555a65347390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.214+0000 7f11d8d89640 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a653478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.215+0000 7f11d958a640 1 -- 172.21.15.102:0/2627246944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 msgr2=0x555a65347e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.215+0000 7f11d958a640 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a65347e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.215+0000 7f11d958a640 1 -- 172.21.15.102:0/2627246944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 msgr2=0x555a653478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.215+0000 7f11d958a640 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a653478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.215+0000 7f11d958a640 1 -- 172.21.15.102:0/2627246944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a65084620 con 0x555a6533f420 2024-04-24T23:26:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.216+0000 7f11d958a640 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 0x555a65347390 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f11d0010370 tx=0x7f11d0010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b3238d4921c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.226+0000 7f11bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:26:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.226+0000 7f11bcff9640 1 -- 172.21.15.102:0/2627246944 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0004440 con 0x555a6533f420 2024-04-24T23:26:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.226+0000 7f11bcff9640 1 -- 172.21.15.102:0/2627246944 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7f11d00035f0 con 0x555a6533f420 2024-04-24T23:26:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.228+0000 7f11de5db740 1 -- 172.21.15.102:0/2627246944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c401c480 con 0x555a6533f420 2024-04-24T23:26:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.228+0000 7f11bcff9640 1 -- 172.21.15.102:0/2627246944 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11d00037f0 con 0x555a6533f420 2024-04-24T23:26:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11de5db740 10 client.16398 fetch_fsmap learned FSMap version 26 2024-04-24T23:26:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11de5db740 1 -- 172.21.15.102:0/2627246944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a65359440 con 0x555a6533f420 2024-04-24T23:26:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11bcff9640 1 -- 172.21.15.102:0/2627246944 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11c401c480 con 0x555a6533f420 2024-04-24T23:26:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11bcff9640 1 -- 172.21.15.102:0/2627246944 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f11d000f040 con 0x555a6533f420 2024-04-24T23:26:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11de5db740 10 client.16398 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:26:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11de5db740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:26:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11de5db740 20 client.16398 populate_metadata read hostname 'smithi102' 2024-04-24T23:26:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11de5db740 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555a6535ad10 0x555a6535d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11de5db740 4 client.16398 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:26:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.229+0000 7f11de5db740 1 -- 172.21.15.102:0/2627246944 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a652b11f0 con 0x555a6535ad10 2024-04-24T23:26:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.233+0000 7f11d8d89640 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555a6535ad10 0x555a6535d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.241+0000 7f11d8d89640 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555a6535ad10 0x555a6535d1b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e969d69ca170f42 server_cookie=19e1b201f954d6c7 in_seq=0 out_seq=0 2024-04-24T23:26:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.248+0000 7f11bcff9640 10 client.16398 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.248+0000 7f11bcff9640 1 -- 172.21.15.102:0/2627246944 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a652b11f0 con 0x555a6535ad10 2024-04-24T23:26:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.248+0000 7f11bcff9640 10 client.16398 handle_command_reply: tid=0 2024-04-24T23:26:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.249+0000 7f11de5db740 1 client.16398 shutdown 2024-04-24T23:26:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.249+0000 7f11de5db740 1 -- 172.21.15.102:0/2627246944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 msgr2=0x555a65347390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.249+0000 7f11de5db740 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 0x555a65347390 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f11d0010370 tx=0x7f11d0010830 comp rx=0 tx=0).stop 2024-04-24T23:26:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.250+0000 7f11de5db740 1 -- 172.21.15.102:0/2627246944 shutdown_connections 2024-04-24T23:26:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.250+0000 7f11de5db740 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a65347e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.250+0000 7f11de5db740 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a653478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.250+0000 7f11de5db740 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555a6535ad10 0x555a6535d1b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.250+0000 7f11de5db740 1 --2- 172.21.15.102:0/2627246944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 0x555a65347390 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.250+0000 7f11de5db740 1 -- 172.21.15.102:0/2627246944 >> 172.21.15.102:0/2627246944 conn(0x555a6533b740 msgr2=0x555a65341540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.250+0000 7f11bcff9640 0 client.16398 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.256+0000 7f11de5db740 1 -- 172.21.15.102:0/2627246944 shutdown_connections 2024-04-24T23:26:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.260+0000 7f11de5db740 1 -- 172.21.15.102:0/2627246944 wait complete. 2024-04-24T23:26:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.260+0000 7f11de5db740 20 client.16398 trim_cache size 0 max 16384 2024-04-24T23:26:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.309+0000 7f11de5db740 1 Processor -- start 2024-04-24T23:26:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.312+0000 7f11de5db740 1 -- start start 2024-04-24T23:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.313+0000 7f11de5db740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 0x555a65349ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.313+0000 7f11de5db740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a653560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.313+0000 7f11de5db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a6533a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.313+0000 7f11de5db740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41c72b0 con 0x555a6533f420 2024-04-24T23:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.313+0000 7f11de5db740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11d411be40 con 0x555a6533fd40 2024-04-24T23:26:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.313+0000 7f11de5db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11d4119cb0 con 0x555a6534a410 2024-04-24T23:26:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.315+0000 7f11d8d89640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a653560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.315+0000 7f11d8d89640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a653560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:38746/0 (socket says 172.21.15.102:38746) 2024-04-24T23:26:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.315+0000 7f11d8d89640 1 -- 172.21.15.102:0/157634932 learned_addr learned my addr 172.21.15.102:0/157634932 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.317+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/157634932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a6533a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.317+0000 7f11d958a640 1 --2- 172.21.15.102:0/157634932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 0x555a65349ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.318+0000 7f11d9d8b640 1 -- 172.21.15.102:0/157634932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 msgr2=0x555a653560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.318+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/157634932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a653560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.318+0000 7f11d9d8b640 1 -- 172.21.15.102:0/157634932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 msgr2=0x555a65349ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.318+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/157634932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 0x555a65349ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.318+0000 7f11d9d8b640 1 -- 172.21.15.102:0/157634932 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a65345900 con 0x555a6534a410 2024-04-24T23:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.320+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/157634932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a6533a720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11c401ce70 tx=0x7f11c4013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a903e94ddb9a927 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.320+0000 7f11beffd640 1 -- 172.21.15.102:0/157634932 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4038070 con 0x555a6534a410 2024-04-24T23:26:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.320+0000 7f11beffd640 1 -- 172.21.15.102:0/157634932 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11c4004c30 con 0x555a6534a410 2024-04-24T23:26:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.322+0000 7f11de5db740 1 -- 172.21.15.102:0/157634932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 msgr2=0x555a6533a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.322+0000 7f11de5db740 1 --2- 172.21.15.102:0/157634932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a6533a720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11c401ce70 tx=0x7f11c4013040 comp rx=0 tx=0).stop 2024-04-24T23:26:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.322+0000 7f11de5db740 1 -- 172.21.15.102:0/157634932 shutdown_connections 2024-04-24T23:26:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.322+0000 7f11de5db740 1 --2- 172.21.15.102:0/157634932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6534a410 0x555a6533a720 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.322+0000 7f11de5db740 1 --2- 172.21.15.102:0/157634932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a653560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.322+0000 7f11de5db740 1 --2- 172.21.15.102:0/157634932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a6533f420 0x555a65349ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.322+0000 7f11de5db740 1 -- 172.21.15.102:0/157634932 >> 172.21.15.102:0/157634932 conn(0x555a6533b740 msgr2=0x555a6533a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.325+0000 7f11de5db740 1 -- 172.21.15.102:0/157634932 shutdown_connections 2024-04-24T23:26:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.325+0000 7f11de5db740 1 -- 172.21.15.102:0/157634932 wait complete. 2024-04-24T23:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.330+0000 7f11de5db740 1 -- start start 2024-04-24T23:26:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.330+0000 7f11de5db740 1 Processor -- start 2024-04-24T23:26:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.331+0000 7f11de5db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533f420 0x555a65344220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.331+0000 7f11de5db740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a65344760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.331+0000 7f11d958a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533f420 0x555a65344220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.331+0000 7f11d958a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533f420 0x555a65344220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:41798/0 (socket says 172.21.15.102:41798) 2024-04-24T23:26:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.331+0000 7f11d958a640 1 -- 172.21.15.102:0/3669035767 learned_addr learned my addr 172.21.15.102:0/3669035767 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.331+0000 7f11d8d89640 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a65344760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.332+0000 7f11de5db740 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a65344ca0 0x555a653c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.332+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a65344ca0 0x555a653c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.332+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11d411a300 con 0x555a65344ca0 2024-04-24T23:26:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.332+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11d411b770 con 0x555a6533fd40 2024-04-24T23:26:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.332+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11d411a9d0 con 0x555a6533f420 2024-04-24T23:26:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.332+0000 7f11de5db740 10 client.0 _ll_register_callbacks cb 0x7f11d0005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:26:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.333+0000 7f11d9d8b640 1 -- 172.21.15.102:0/3669035767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533f420 msgr2=0x555a65344220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.333+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533f420 0x555a65344220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.333+0000 7f11d9d8b640 1 -- 172.21.15.102:0/3669035767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 msgr2=0x555a65344760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:26:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.333+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a65344760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.333+0000 7f11d9d8b640 1 -- 172.21.15.102:0/3669035767 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a653c08f0 con 0x555a65344ca0 2024-04-24T23:26:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.333+0000 7f11d9d8b640 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a65344ca0 0x555a653c0330 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f11c40104a0 tx=0x7f11c4005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f702beba4e6860eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.334+0000 7f11bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-24T23:26:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.335+0000 7f11bcff9640 1 -- 172.21.15.102:0/3669035767 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4038040 con 0x555a65344ca0 2024-04-24T23:26:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.335+0000 7f11bcff9640 1 -- 172.21.15.102:0/3669035767 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7f11c4004780 con 0x555a65344ca0 2024-04-24T23:26:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.335+0000 7f11d958a640 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533f420 0x555a65344220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.335+0000 7f11d8d89640 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a65344760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.335+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555a652ab540 con 0x555a65344ca0 2024-04-24T23:26:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.341+0000 7f11bcff9640 1 -- 172.21.15.102:0/3669035767 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11c404a050 con 0x555a65344ca0 2024-04-24T23:26:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.349+0000 7f11de5db740 10 client.16404 fetch_fsmap learned FSMap version 26 2024-04-24T23:26:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.349+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a653c1c30 con 0x555a65344ca0 2024-04-24T23:26:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.353+0000 7f11bcff9640 1 -- 172.21.15.102:0/3669035767 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11c4027450 con 0x555a65344ca0 2024-04-24T23:26:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.353+0000 7f11bcff9640 1 -- 172.21.15.102:0/3669035767 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f11c4033400 con 0x555a65344ca0 2024-04-24T23:26:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.353+0000 7f11de5db740 10 client.16404 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T23:26:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.353+0000 7f11de5db740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T23:26:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.354+0000 7f11de5db740 20 client.16404 populate_metadata read hostname 'smithi102' 2024-04-24T23:26:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.354+0000 7f11de5db740 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555a653c2270 0x555a653c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.354+0000 7f11de5db740 4 client.16404 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T23:26:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.354+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 --> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a652ab540 con 0x555a653c2270 2024-04-24T23:26:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.358+0000 7f11d958a640 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555a653c2270 0x555a653c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.359+0000 7f11d958a640 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555a653c2270 0x555a653c4690 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4385dd3184e390ab server_cookie=520a2d72ec35e0f in_seq=0 out_seq=0 2024-04-24T23:26:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.363+0000 7f11bcff9640 10 client.16404 ms_handle_connect on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.508+0000 7f11bcff9640 1 -- 172.21.15.102:0/3669035767 <== mds.0 v2:172.21.15.161:6838/1258581930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a652ab540 con 0x555a653c2270 2024-04-24T23:26:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.508+0000 7f11bcff9640 10 client.16404 handle_command_reply: tid=0 2024-04-24T23:26:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.508+0000 7f11de5db740 1 client.16404 shutdown 2024-04-24T23:26:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.509+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a65344ca0 msgr2=0x555a653c0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.509+0000 7f11de5db740 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a65344ca0 0x555a653c0330 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f11c40104a0 tx=0x7f11c4005690 comp rx=0 tx=0).stop 2024-04-24T23:26:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.509+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 shutdown_connections 2024-04-24T23:26:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.509+0000 7f11de5db740 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.161:6838/1258581930,v1:172.21.15.161:6839/1258581930] conn(0x555a653c2270 0x555a653c4690 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.509+0000 7f11de5db740 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a65344ca0 0x555a653c0330 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.509+0000 7f11de5db740 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a6533fd40 0x555a65344760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.509+0000 7f11de5db740 1 --2- 172.21.15.102:0/3669035767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6533f420 0x555a65344220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.509+0000 7f11bcff9640 0 client.16404 ms_handle_reset on v2:172.21.15.161:6838/1258581930 2024-04-24T23:26:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.509+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 >> 172.21.15.102:0/3669035767 conn(0x555a6533b740 msgr2=0x555a6533a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:45 smithi183 ceph-mon[33137]: pgmap v428: 97 pgs: 97 active+clean; 19 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 23 MiB/s wr, 5.87k op/s 2024-04-24T23:26:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:{ 2024-04-24T23:26:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "message": "", 2024-04-24T23:26:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout: "return_code": 0 2024-04-24T23:26:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stdout:} 2024-04-24T23:26:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.511+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 shutdown_connections 2024-04-24T23:26:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.511+0000 7f11de5db740 1 -- 172.21.15.102:0/3669035767 wait complete. 2024-04-24T23:26:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.511+0000 7f11de5db740 20 client.16404 trim_cache size 0 max 16384 2024-04-24T23:26:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.516+0000 7f11da58c640 1 -- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11c00a3130 msgr2=0x7f11c00a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.516+0000 7f11da58c640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11c00a3130 0x7f11c00a55f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11c4010550 tx=0x7f11c400bf90 comp rx=0 tx=0).stop 2024-04-24T23:26:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.516+0000 7f11da58c640 1 -- 172.21.15.102:0/924749408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d4100160 msgr2=0x7f11d4109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.516+0000 7f11da58c640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d4100160 0x7f11d4109620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11cc035040 tx=0x7f11cc00a300 comp rx=0 tx=0).stop 2024-04-24T23:26:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.523+0000 7f11da58c640 1 -- 172.21.15.102:0/924749408 shutdown_connections 2024-04-24T23:26:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.523+0000 7f11da58c640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f11c00a3130 0x7f11c00a55f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.523+0000 7f11da58c640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11d41057b0 0x7f11d4109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.523+0000 7f11da58c640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d4100160 0x7f11d4109620 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.523+0000 7f11da58c640 1 --2- 172.21.15.102:0/924749408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d40ff7a0 0x7f11d410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.523+0000 7f11da58c640 1 -- 172.21.15.102:0/924749408 >> 172.21.15.102:0/924749408 conn(0x7f11d40765a0 msgr2=0x7f11d4102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.523+0000 7f11da58c640 1 -- 172.21.15.102:0/924749408 shutdown_connections 2024-04-24T23:26:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi102.front.sepia.ceph.com.smithi102.stderr:2024-04-24T23:26:45.523+0000 7f11da58c640 1 -- 172.21.15.102:0/924749408 wait complete. 2024-04-24T23:26:45.575 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:45 smithi102 ceph-mon[29216]: pgmap v428: 97 pgs: 97 active+clean; 19 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 23 MiB/s wr, 5.87k op/s 2024-04-24T23:26:45.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:45 smithi161 ceph-mon[32709]: pgmap v428: 97 pgs: 97 active+clean; 19 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 23 MiB/s wr, 5.87k op/s 2024-04-24T23:26:47.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:47 smithi183 ceph-mon[33137]: pgmap v429: 97 pgs: 97 active+clean; 19 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 19 MiB/s wr, 4.94k op/s 2024-04-24T23:26:47.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:47 smithi102 ceph-mon[29216]: pgmap v429: 97 pgs: 97 active+clean; 19 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 19 MiB/s wr, 4.94k op/s 2024-04-24T23:26:47.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:47 smithi161 ceph-mon[32709]: pgmap v429: 97 pgs: 97 active+clean; 19 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 19 MiB/s wr, 4.94k op/s 2024-04-24T23:26:49.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:49 smithi183 ceph-mon[33137]: pgmap v430: 97 pgs: 97 active+clean; 19 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2024-04-24T23:26:49.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:49 smithi102 ceph-mon[29216]: pgmap v430: 97 pgs: 97 active+clean; 19 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2024-04-24T23:26:49.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:49 smithi161 ceph-mon[32709]: pgmap v430: 97 pgs: 97 active+clean; 19 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2024-04-24T23:26:51.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:51 smithi183 ceph-mon[33137]: pgmap v431: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 23 MiB/s wr, 5.80k op/s 2024-04-24T23:26:51.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:51 smithi102 ceph-mon[29216]: pgmap v431: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 23 MiB/s wr, 5.80k op/s 2024-04-24T23:26:51.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:51 smithi161 ceph-mon[32709]: pgmap v431: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 23 MiB/s wr, 5.80k op/s 2024-04-24T23:26:53.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:53 smithi183 ceph-mon[33137]: pgmap v432: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2024-04-24T23:26:53.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:53 smithi102 ceph-mon[29216]: pgmap v432: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2024-04-24T23:26:53.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:53 smithi161 ceph-mon[32709]: pgmap v432: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2024-04-24T23:26:53.821 INFO:tasks.workunit.client.0.smithi102.stdout:Syncing()...4 sec 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout:FFSB benchmark finished at: Wed Apr 24 23:26:53 2024 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout:Results: 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout:Benchmark took 308.88 sec 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout:Total Results 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout:=============== 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout: ======= ============ ========= ======= =========== ========== 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout: write : 1844480 5971.43 100.000% 100.000% 23.3MB/sec 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout:- 2024-04-24T23:26:53.822 INFO:tasks.workunit.client.0.smithi102.stdout:5971.43 Transactions per Second 2024-04-24T23:26:53.823 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.823 INFO:tasks.workunit.client.0.smithi102.stdout:Throughput Results 2024-04-24T23:26:53.823 INFO:tasks.workunit.client.0.smithi102.stdout:=================== 2024-04-24T23:26:53.823 INFO:tasks.workunit.client.0.smithi102.stdout:Write Throughput: 23.3MB/sec 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout:System Call Latency statistics in millisecs 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout:===== 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout: Min Avg Max Total Calls 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout: ======== ======== ======== ============ 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout:[ open] 0.505000 49.602559 10591.839844 1441 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[0] 5000.000000 - 10000.000000 : 3 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[1] 2000.000000 - 5000.000000 : 3 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[2] 1000.000000 - 2000.000000 : 1 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[4] 200.000000 - 500.000000 : 1 2024-04-24T23:26:53.830 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[5] 100.000000 - 200.000000 : 2 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[6] 50.000000 - 100.000000 : 35 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[7] 20.000000 - 50.000000 : 394 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[8] 10.000000 - 20.000000 : 651 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[9] 5.000000 - 10.000000 : 228 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[10] 2.000000 - 5.000000 : 72 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[11] 1.000000 - 2.000000 : 38 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[12] 0.500000 - 1.000000 : 12 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout:[ write] 0.088000 5.217438 6242.075195 1844480 2024-04-24T23:26:53.831 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[0] 5000.000000 - 10000.000000 : 2 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[2] 1000.000000 - 2000.000000 : 9 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[3] 500.000000 - 1000.000000 : 17 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[4] 200.000000 - 500.000000 : 274 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[5] 100.000000 - 200.000000 : 1772 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[6] 50.000000 - 100.000000 : 7052 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[7] 20.000000 - 50.000000 : 65867 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[8] 10.000000 - 20.000000 : 171369 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[9] 5.000000 - 10.000000 : 341562 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[10] 2.000000 - 5.000000 : 530851 2024-04-24T23:26:53.832 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[11] 1.000000 - 2.000000 : 186157 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[12] 0.500000 - 1.000000 : 45389 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[13] 0.200000 - 0.500000 : 240571 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[14] 0.100000 - 0.200000 : 252975 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[15] 0.050000 - 0.100000 : 613 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout:[ lseek] 0.000000 0.000840 1.801000 1844480 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-24T23:26:53.833 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[10] 2.000000 - 5.000000 : 0 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[11] 1.000000 - 2.000000 : 1 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[12] 0.500000 - 1.000000 : 6 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[13] 0.200000 - 0.500000 : 10 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[14] 0.100000 - 0.200000 : 16 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[15] 0.050000 - 0.100000 : 39 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[16] 0.020000 - 0.050000 : 336 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[17] 0.010000 - 0.020000 : 1406 2024-04-24T23:26:53.834 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[18] 0.000000 - 0.010000 : 1842666 2024-04-24T23:26:53.835 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.835 INFO:tasks.workunit.client.0.smithi102.stdout:[ close] 0.021000 2.611522 156.580994 1441 2024-04-24T23:26:53.835 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[5] 100.000000 - 200.000000 : 2 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[6] 50.000000 - 100.000000 : 3 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[7] 20.000000 - 50.000000 : 27 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[8] 10.000000 - 20.000000 : 59 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[9] 5.000000 - 10.000000 : 95 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[10] 2.000000 - 5.000000 : 181 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[11] 1.000000 - 2.000000 : 179 2024-04-24T23:26:53.838 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[12] 0.500000 - 1.000000 : 98 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[13] 0.200000 - 0.500000 : 34 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[14] 0.100000 - 0.200000 : 161 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[15] 0.050000 - 0.100000 : 132 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[16] 0.020000 - 0.050000 : 470 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout: 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout:1.0% User Time 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout:30.5% System Time 2024-04-24T23:26:53.839 INFO:tasks.workunit.client.0.smithi102.stdout:31.5% CPU Utilization 2024-04-24T23:26:53.840 INFO:tasks.workunit.client.0.smithi102.stderr:+ cd .. 2024-04-24T23:26:53.846 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm -r tmp ffsb 2024-04-24T23:26:55.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:55 smithi183 ceph-mon[33137]: pgmap v433: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 5.10k op/s 2024-04-24T23:26:55.570 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:55 smithi102 ceph-mon[29216]: pgmap v433: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 5.10k op/s 2024-04-24T23:26:55.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:55 smithi161 ceph-mon[32709]: pgmap v433: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 5.10k op/s 2024-04-24T23:26:57.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:57 smithi183 ceph-mon[33137]: pgmap v434: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-04-24T23:26:57.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:57 smithi102 ceph-mon[29216]: pgmap v434: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-04-24T23:26:57.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:57 smithi161 ceph-mon[32709]: pgmap v434: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 3.54k op/s 2024-04-24T23:26:57.846 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T23:26:57.847 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T23:26:58.025 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-04-24T23:26:58.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T23:26:58.472 DEBUG:teuthology.parallel:result is None 2024-04-24T23:26:58.473 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:26:58.664 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:26:58.665 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-24T23:26:58.676 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:26:58.685 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-24T23:26:58.694 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-24T23:26:58.694 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T23:26:58.695 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:26:58.695 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-24T23:26:58.772 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-24T23:26:58.822 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T23:26:58.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.981+0000 7efe3891d640 1 -- 172.21.15.102:0/3570977703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe34107fc0 msgr2=0x7efe341083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.981+0000 7efe3891d640 1 --2- 172.21.15.102:0/3570977703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe34107fc0 0x7efe341083a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe28009900 tx=0x7efe2802a410 comp rx=0 tx=0).stop 2024-04-24T23:26:58.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.982+0000 7efe3891d640 1 -- 172.21.15.102:0/3570977703 shutdown_connections 2024-04-24T23:26:58.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.982+0000 7efe3891d640 1 --2- 172.21.15.102:0/3570977703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3410d090 0x7efe34114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.982+0000 7efe3891d640 1 --2- 172.21.15.102:0/3570977703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34108970 0x7efe3410c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.982+0000 7efe3891d640 1 --2- 172.21.15.102:0/3570977703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe34107fc0 0x7efe341083a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.982+0000 7efe3891d640 1 -- 172.21.15.102:0/3570977703 >> 172.21.15.102:0/3570977703 conn(0x7efe34075cc0 msgr2=0x7efe340780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:58.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.983+0000 7efe3891d640 1 -- 172.21.15.102:0/3570977703 shutdown_connections 2024-04-24T23:26:58.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.983+0000 7efe3891d640 1 -- 172.21.15.102:0/3570977703 wait complete. 2024-04-24T23:26:58.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3891d640 1 Processor -- start 2024-04-24T23:26:58.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3891d640 1 -- start start 2024-04-24T23:26:58.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3891d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34107fc0 0x7efe3419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3891d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3410d090 0x7efe3419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3891d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe341a2e20 0x7efe341a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3891d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efe34118050 con 0x7efe341a2e20 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3891d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe34117ed0 con 0x7efe3410d090 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3891d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe341181d0 con 0x7efe34107fc0 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34107fc0 0x7efe3419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34107fc0 0x7efe3419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:39960/0 (socket says 172.21.15.102:39960) 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe3259c640 1 -- 172.21.15.102:0/532833998 learned_addr learned my addr 172.21.15.102:0/532833998 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.984+0000 7efe31d9b640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3410d090 0x7efe3419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.985+0000 7efe32d9d640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe341a2e20 0x7efe341a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.985+0000 7efe3259c640 1 -- 172.21.15.102:0/532833998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3410d090 msgr2=0x7efe3419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.985+0000 7efe3259c640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3410d090 0x7efe3419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.985+0000 7efe3259c640 1 -- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe341a2e20 msgr2=0x7efe341a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.985+0000 7efe3259c640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe341a2e20 0x7efe341a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:58.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.985+0000 7efe3259c640 1 -- 172.21.15.102:0/532833998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe341a3910 con 0x7efe34107fc0 2024-04-24T23:26:58.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.985+0000 7efe32d9d640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe341a2e20 0x7efe341a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:58.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.985+0000 7efe3259c640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34107fc0 0x7efe3419e5b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe28006010 tx=0x7efe280045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:58.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.986+0000 7efe237fe640 1 -- 172.21.15.102:0/532833998 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28040070 con 0x7efe34107fc0 2024-04-24T23:26:58.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.986+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe341a3ba0 con 0x7efe34107fc0 2024-04-24T23:26:58.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.986+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe341ab360 con 0x7efe34107fc0 2024-04-24T23:26:58.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.986+0000 7efe237fe640 1 -- 172.21.15.102:0/532833998 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe280316d0 con 0x7efe34107fc0 2024-04-24T23:26:58.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.986+0000 7efe237fe640 1 -- 172.21.15.102:0/532833998 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28035020 con 0x7efe34107fc0 2024-04-24T23:26:58.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.987+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe14005290 con 0x7efe34107fc0 2024-04-24T23:26:58.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.989+0000 7efe237fe640 1 -- 172.21.15.102:0/532833998 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7efe28043070 con 0x7efe34107fc0 2024-04-24T23:26:58.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.990+0000 7efe237fe640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efe1c0a2f00 0x7efe1c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:58.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.990+0000 7efe31d9b640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efe1c0a2f00 0x7efe1c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:58.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.990+0000 7efe237fe640 1 -- 172.21.15.102:0/532833998 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7efe280efd00 con 0x7efe34107fc0 2024-04-24T23:26:58.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.990+0000 7efe31d9b640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efe1c0a2f00 0x7efe1c0a53c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efe340760e0 tx=0x7efe2c00d040 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:58.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:58.994+0000 7efe237fe640 1 -- 172.21.15.102:0/532833998 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe280bc8e0 con 0x7efe34107fc0 2024-04-24T23:26:59.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.357+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 --> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7efe14000bc0 con 0x7efe1c0a2f00 2024-04-24T23:26:59.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.374+0000 7efe237fe640 1 -- 172.21.15.102:0/532833998 <== mgr.15288 v2:172.21.15.102:6800/161836032 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7efe14000bc0 con 0x7efe1c0a2f00 2024-04-24T23:26:59.375 INFO:teuthology.orchestra.run.smithi102.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T23:13:03", "first": "2024-04-24T23:14:00", "last": "2024-04-24T23:26:00", "last_pruned": "2024-04-24T23:26:01", "created_count": 13, "pruned_count": 7, "active": true} 2024-04-24T23:26:59.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.377+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efe1c0a2f00 msgr2=0x7efe1c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.377+0000 7efe3891d640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efe1c0a2f00 0x7efe1c0a53c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efe340760e0 tx=0x7efe2c00d040 comp rx=0 tx=0).stop 2024-04-24T23:26:59.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.377+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34107fc0 msgr2=0x7efe3419e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.377+0000 7efe3891d640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34107fc0 0x7efe3419e5b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe28006010 tx=0x7efe280045e0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.377+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 shutdown_connections 2024-04-24T23:26:59.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.377+0000 7efe3891d640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7efe1c0a2f00 0x7efe1c0a53c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.377+0000 7efe3891d640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe341a2e20 0x7efe341a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.377+0000 7efe3891d640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe3410d090 0x7efe3419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.377+0000 7efe3891d640 1 --2- 172.21.15.102:0/532833998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34107fc0 0x7efe3419e5b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.378+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 >> 172.21.15.102:0/532833998 conn(0x7efe34075cc0 msgr2=0x7efe34078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:59.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.378+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 shutdown_connections 2024-04-24T23:26:59.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.378+0000 7efe3891d640 1 -- 172.21.15.102:0/532833998 wait complete. 2024-04-24T23:26:59.395 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T23:26:59.416 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:26:59 smithi102 ceph-mon[29216]: pgmap v435: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-04-24T23:26:59.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:26:59 smithi183 ceph-mon[33137]: pgmap v435: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-04-24T23:26:59.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.557+0000 7f253fe9e640 1 -- 172.21.15.102:0/2701791321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 msgr2=0x7f2538108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.557+0000 7f253fe9e640 1 --2- 172.21.15.102:0/2701791321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 0x7f2538108740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2530009900 tx=0x7f253002a4a0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.557+0000 7f253fe9e640 1 -- 172.21.15.102:0/2701791321 shutdown_connections 2024-04-24T23:26:59.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.557+0000 7f253fe9e640 1 --2- 172.21.15.102:0/2701791321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f253811af20 0x7f253811d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.557+0000 7f253fe9e640 1 --2- 172.21.15.102:0/2701791321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2538110310 0x7f2538108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.557+0000 7f253fe9e640 1 --2- 172.21.15.102:0/2701791321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 0x7f2538108740 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.557+0000 7f253fe9e640 1 -- 172.21.15.102:0/2701791321 >> 172.21.15.102:0/2701791321 conn(0x7f253806c810 msgr2=0x7f253806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:59.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.558+0000 7f253fe9e640 1 -- 172.21.15.102:0/2701791321 shutdown_connections 2024-04-24T23:26:59.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.558+0000 7f253fe9e640 1 -- 172.21.15.102:0/2701791321 wait complete. 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.558+0000 7f253fe9e640 1 Processor -- start 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.558+0000 7f253fe9e640 1 -- start start 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.558+0000 7f253fe9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 0x7f25381a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.558+0000 7f253fe9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f253811af20 0x7f25381a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253fe9e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25381ab680 0x7f25381abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253dc3a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 0x7f25381a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253dc3a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 0x7f25381a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:39974/0 (socket says 172.21.15.102:39974) 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253fe9e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2538120840 con 0x7f25381ab680 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253fe9e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f25381206c0 con 0x7f253811af20 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253d439640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f253811af20 0x7f25381a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253fe9e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25381209c0 con 0x7f253810ff30 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253d439640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f253811af20 0x7f25381a7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:34166/0 (socket says 172.21.15.102:34166) 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253dc3a640 1 -- 172.21.15.102:0/3356746439 learned_addr learned my addr 172.21.15.102:0/3356746439 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:26:59.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253e43b640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25381ab680 0x7f25381abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253dc3a640 1 -- 172.21.15.102:0/3356746439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f253811af20 msgr2=0x7f25381a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253dc3a640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f253811af20 0x7f25381a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253dc3a640 1 -- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25381ab680 msgr2=0x7f25381abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253dc3a640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25381ab680 0x7f25381abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.559+0000 7f253dc3a640 1 -- 172.21.15.102:0/3356746439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25381ac170 con 0x7f253810ff30 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.560+0000 7f253dc3a640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 0x7f25381a6dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2530005ab0 tx=0x7f2530031b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.560+0000 7f2536ffd640 1 -- 172.21.15.102:0/3356746439 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530040030 con 0x7f253810ff30 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.560+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25381ac400 con 0x7f253810ff30 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.560+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25381b3c20 con 0x7f253810ff30 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.560+0000 7f2536ffd640 1 -- 172.21.15.102:0/3356746439 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f253003c8f0 con 0x7f253810ff30 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.560+0000 7f2536ffd640 1 -- 172.21.15.102:0/3356746439 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530044920 con 0x7f253810ff30 2024-04-24T23:26:59.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.561+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2524005290 con 0x7f253810ff30 2024-04-24T23:26:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.562+0000 7f2536ffd640 1 -- 172.21.15.102:0/3356746439 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f2530043030 con 0x7f253810ff30 2024-04-24T23:26:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.563+0000 7f2536ffd640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f25200a2e20 0x7f25200a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.563+0000 7f2536ffd640 1 -- 172.21.15.102:0/3356746439 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7f25300ef930 con 0x7f253810ff30 2024-04-24T23:26:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.563+0000 7f253d439640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f25200a2e20 0x7f25200a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:59.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.563+0000 7f253d439640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f25200a2e20 0x7f25200a52e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f253810f100 tx=0x7f2528009680 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:59.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.567+0000 7f2536ffd640 1 -- 172.21.15.102:0/3356746439 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f25300bc510 con 0x7f253810ff30 2024-04-24T23:26:59.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:26:59 smithi161 ceph-mon[32709]: pgmap v435: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-04-24T23:26:59.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:26:59.934+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 --> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2524000bc0 con 0x7f25200a2e20 2024-04-24T23:27:00.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.000+0000 7f2536ffd640 1 -- 172.21.15.102:0/3356746439 <== mgr.15288 v2:172.21.15.102:6800/161836032 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2524000bc0 con 0x7f25200a2e20 2024-04-24T23:27:00.002 INFO:teuthology.orchestra.run.smithi102.stdout:/ 1m 6m3h 2024-04-24T23:27:00.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f25200a2e20 msgr2=0x7f25200a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f25200a2e20 0x7f25200a52e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f253810f100 tx=0x7f2528009680 comp rx=0 tx=0).stop 2024-04-24T23:27:00.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 msgr2=0x7f25381a6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 0x7f25381a6dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2530005ab0 tx=0x7f2530031b30 comp rx=0 tx=0).stop 2024-04-24T23:27:00.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 shutdown_connections 2024-04-24T23:27:00.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f25200a2e20 0x7f25200a52e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25381ab680 0x7f25381abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f253811af20 0x7f25381a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 --2- 172.21.15.102:0/3356746439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253810ff30 0x7f25381a6dc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 >> 172.21.15.102:0/3356746439 conn(0x7f253806c810 msgr2=0x7f253810dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:00.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 shutdown_connections 2024-04-24T23:27:00.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.003+0000 7f253fe9e640 1 -- 172.21.15.102:0/3356746439 wait complete. 2024-04-24T23:27:00.015 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-24T23:27:00.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:27:00.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.217+0000 7fdbd8a31640 1 -- 172.21.15.102:0/479505008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 msgr2=0x7fdbd410da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.217+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/479505008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 0x7fdbd410da80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009860 tx=0x7fdbbc02a340 comp rx=0 tx=0).stop 2024-04-24T23:27:00.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.217+0000 7fdbd8a31640 1 -- 172.21.15.102:0/479505008 shutdown_connections 2024-04-24T23:27:00.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.217+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/479505008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd410e6a0 0x7fdbd411cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.217+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/479505008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd4110310 0x7fdbd410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.217+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/479505008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 0x7fdbd410da80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.217+0000 7fdbd8a31640 1 -- 172.21.15.102:0/479505008 >> 172.21.15.102:0/479505008 conn(0x7fdbd406c860 msgr2=0x7fdbd406cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:00.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.218+0000 7fdbd8a31640 1 -- 172.21.15.102:0/479505008 shutdown_connections 2024-04-24T23:27:00.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.218+0000 7fdbd8a31640 1 -- 172.21.15.102:0/479505008 wait complete. 2024-04-24T23:27:00.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.218+0000 7fdbd8a31640 1 Processor -- start 2024-04-24T23:27:00.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd8a31640 1 -- start start 2024-04-24T23:27:00.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd8a31640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 0x7fdbd41a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd8a31640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd41a72d0 0x7fdbd41ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd8a31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd41abb90 0x7fdbd41ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd8a31640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4120780 con 0x7fdbd41a72d0 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd8a31640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4120600 con 0x7fdbd410fea0 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd8a31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4120900 con 0x7fdbd41abb90 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 0x7fdbd41a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd1d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd41a72d0 0x7fdbd41ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 0x7fdbd41a6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.102:34170/0 (socket says 172.21.15.102:34170) 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd259c640 1 -- 172.21.15.102:0/1042597902 learned_addr learned my addr 172.21.15.102:0/1042597902 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd1d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd41a72d0 0x7fdbd41ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33986/0 (socket says 172.21.15.102:33986) 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.219+0000 7fdbd2d9d640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd41abb90 0x7fdbd41ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.220+0000 7fdbd1d9b640 1 -- 172.21.15.102:0/1042597902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd41abb90 msgr2=0x7fdbd41ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.220+0000 7fdbd1d9b640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd41abb90 0x7fdbd41ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.220+0000 7fdbd1d9b640 1 -- 172.21.15.102:0/1042597902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 msgr2=0x7fdbd41a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.220+0000 7fdbd1d9b640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 0x7fdbd41a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.220+0000 7fdbd1d9b640 1 -- 172.21.15.102:0/1042597902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd4110700 con 0x7fdbd41a72d0 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.220+0000 7fdbd259c640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 0x7fdbd41a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.220+0000 7fdbd1d9b640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd41a72d0 0x7fdbd41ab650 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400ef60 tx=0x7fdbc400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.220+0000 7fdbcb7fe640 1 -- 172.21.15.102:0/1042597902 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc401b070 con 0x7fdbd41a72d0 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.221+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd41109f0 con 0x7fdbd41a72d0 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.221+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd4110f40 con 0x7fdbd41a72d0 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.221+0000 7fdbcb7fe640 1 -- 172.21.15.102:0/1042597902 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbc400ba20 con 0x7fdbd41a72d0 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.221+0000 7fdbcb7fe640 1 -- 172.21.15.102:0/1042597902 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4016bf0 con 0x7fdbd41a72d0 2024-04-24T23:27:00.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.222+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb4005290 con 0x7fdbd41a72d0 2024-04-24T23:27:00.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.223+0000 7fdbcb7fe640 1 -- 172.21.15.102:0/1042597902 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fdbc401e070 con 0x7fdbd41a72d0 2024-04-24T23:27:00.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.224+0000 7fdbcb7fe640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdbc00a2ed0 0x7fdbc00a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.224+0000 7fdbcb7fe640 1 -- 172.21.15.102:0/1042597902 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7fdbc40cf180 con 0x7fdbd41a72d0 2024-04-24T23:27:00.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.224+0000 7fdbd259c640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdbc00a2ed0 0x7fdbc00a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.226+0000 7fdbd259c640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdbc00a2ed0 0x7fdbc00a5390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0095f0 tx=0x7fdbbc030340 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.234+0000 7fdbcb7fe640 1 -- 172.21.15.102:0/1042597902 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdbc409b6b0 con 0x7fdbd41a72d0 2024-04-24T23:27:00.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:00 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:00.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:00 smithi102 ceph-mon[29216]: from='client.25759 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:00.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:00 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:00.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:00 smithi183 ceph-mon[33137]: from='client.25759 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:00.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.572+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbb4005850 con 0x7fdbd41a72d0 2024-04-24T23:27:00.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.574+0000 7fdbcb7fe640 1 -- 172.21.15.102:0/1042597902 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+23243 (secure 0 0 0) 0x7fdbc409f9b0 con 0x7fdbd41a72d0 2024-04-24T23:27:00.576 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:27:00.576 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":105,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:26:30.058147+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/2090298312":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/835175442":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/3611055373":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/4149172465":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/1812533171":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/3611055373":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:27:00.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdbc00a2ed0 msgr2=0x7fdbc00a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdbc00a2ed0 0x7fdbc00a5390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0095f0 tx=0x7fdbbc030340 comp rx=0 tx=0).stop 2024-04-24T23:27:00.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd41a72d0 msgr2=0x7fdbd41ab650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd41a72d0 0x7fdbd41ab650 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400ef60 tx=0x7fdbc400ef90 comp rx=0 tx=0).stop 2024-04-24T23:27:00.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 shutdown_connections 2024-04-24T23:27:00.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd41abb90 0x7fdbd41ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdbc00a2ed0 0x7fdbc00a5390 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0095f0 tx=0x7fdbbc030340 comp rx=0 tx=0).stop 2024-04-24T23:27:00.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd41a72d0 0x7fdbd41ab650 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 --2- 172.21.15.102:0/1042597902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd410fea0 0x7fdbd41a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 >> 172.21.15.102:0/1042597902 conn(0x7fdbd406c860 msgr2=0x7fdbd41199c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:00.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 shutdown_connections 2024-04-24T23:27:00.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.578+0000 7fdbd8a31640 1 -- 172.21.15.102:0/1042597902 wait complete. 2024-04-24T23:27:00.598 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:58.730586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:59.855279+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:27:00.598 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:27:00.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:00 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:00.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:00 smithi161 ceph-mon[32709]: from='client.25759 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.762+0000 7f7453d42640 1 -- 172.21.15.102:0/3581476316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f744c10dfc0 msgr2=0x7f744c10e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.762+0000 7f7453d42640 1 --2- 172.21.15.102:0/3581476316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f744c10dfc0 0x7f744c10e440 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7448009cd0 tx=0x7f744802a930 comp rx=0 tx=0).stop 2024-04-24T23:27:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.764+0000 7f7453d42640 1 -- 172.21.15.102:0/3581476316 shutdown_connections 2024-04-24T23:27:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.764+0000 7f7453d42640 1 --2- 172.21.15.102:0/3581476316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c11af20 0x7f744c11d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.764+0000 7f7453d42640 1 --2- 172.21.15.102:0/3581476316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f744c10dfc0 0x7f744c10e440 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.764+0000 7f7453d42640 1 --2- 172.21.15.102:0/3581476316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c10ff90 0x7f744c10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.764+0000 7f7453d42640 1 -- 172.21.15.102:0/3581476316 >> 172.21.15.102:0/3581476316 conn(0x7f744c06c850 msgr2=0x7f744c06cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:00.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.770+0000 7f7453d42640 1 -- 172.21.15.102:0/3581476316 shutdown_connections 2024-04-24T23:27:00.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.772+0000 7f7453d42640 1 -- 172.21.15.102:0/3581476316 wait complete. 2024-04-24T23:27:00.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.773+0000 7f7453d42640 1 Processor -- start 2024-04-24T23:27:00.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.773+0000 7f7453d42640 1 -- start start 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.773+0000 7f7453d42640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c10dfc0 0x7f744c1a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.773+0000 7f7453d42640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c10ff90 0x7f744c1a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.773+0000 7f7453d42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f744c11af20 0x7f744c1ab670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f74512dd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c10ff90 0x7f744c1a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f74512dd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c10ff90 0x7f744c1a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:40004/0 (socket says 172.21.15.102:40004) 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f74512dd640 1 -- 172.21.15.102:0/2901209184 learned_addr learned my addr 172.21.15.102:0/2901209184 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f744c120830 con 0x7f744c11af20 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f744c1206b0 con 0x7f744c10dfc0 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f744c1209b0 con 0x7f744c10ff90 2024-04-24T23:27:00.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f74522df640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f744c11af20 0x7f744c1ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f7451ade640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c10dfc0 0x7f744c1a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f74512dd640 1 -- 172.21.15.102:0/2901209184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c10dfc0 msgr2=0x7f744c1a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f74512dd640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c10dfc0 0x7f744c1a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f74512dd640 1 -- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f744c11af20 msgr2=0x7f744c1ab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:00.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.774+0000 7f74512dd640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f744c11af20 0x7f744c1ab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:00.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.775+0000 7f74512dd640 1 -- 172.21.15.102:0/2901209184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c1abda0 con 0x7f744c10ff90 2024-04-24T23:27:00.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.775+0000 7f74512dd640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c10ff90 0x7f744c1a72f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7448004ae0 tx=0x7f744802f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.775+0000 7f7442ffd640 1 -- 172.21.15.102:0/2901209184 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448003cb0 con 0x7f744c10ff90 2024-04-24T23:27:00.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.775+0000 7f7442ffd640 1 -- 172.21.15.102:0/2901209184 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f744802d070 con 0x7f744c10ff90 2024-04-24T23:27:00.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.775+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744c1ac030 con 0x7f744c10ff90 2024-04-24T23:27:00.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.775+0000 7f7442ffd640 1 -- 172.21.15.102:0/2901209184 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744802f9a0 con 0x7f744c10ff90 2024-04-24T23:27:00.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.775+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744c1b3920 con 0x7f744c10ff90 2024-04-24T23:27:00.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.778+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7430005290 con 0x7f744c10ff90 2024-04-24T23:27:00.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.778+0000 7f7442ffd640 1 -- 172.21.15.102:0/2901209184 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f744802fb40 con 0x7f744c10ff90 2024-04-24T23:27:00.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.779+0000 7f7442ffd640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f74380a2d40 0x7f74380a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:00.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.779+0000 7f7442ffd640 1 -- 172.21.15.102:0/2901209184 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7f74480f0140 con 0x7f744c10ff90 2024-04-24T23:27:00.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.780+0000 7f7451ade640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f74380a2d40 0x7f74380a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:00.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.780+0000 7f7451ade640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f74380a2d40 0x7f74380a5200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7444006fd0 tx=0x7f7444008040 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:00.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:00.786+0000 7f7442ffd640 1 -- 172.21.15.102:0/2901209184 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f74480bce30 con 0x7f744c10ff90 2024-04-24T23:27:01.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.081+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7430005850 con 0x7f744c10ff90 2024-04-24T23:27:01.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.081+0000 7f7442ffd640 1 -- 172.21.15.102:0/2901209184 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f74480a70f0 con 0x7f744c10ff90 2024-04-24T23:27:01.083 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:27:01.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f74380a2d40 msgr2=0x7f74380a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f74380a2d40 0x7f74380a5200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7444006fd0 tx=0x7f7444008040 comp rx=0 tx=0).stop 2024-04-24T23:27:01.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c10ff90 msgr2=0x7f744c1a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c10ff90 0x7f744c1a72f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7448004ae0 tx=0x7f744802f410 comp rx=0 tx=0).stop 2024-04-24T23:27:01.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 shutdown_connections 2024-04-24T23:27:01.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f74380a2d40 0x7f74380a5200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f744c11af20 0x7f744c1ab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f744c10ff90 0x7f744c1a72f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 --2- 172.21.15.102:0/2901209184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c10dfc0 0x7f744c1a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 >> 172.21.15.102:0/2901209184 conn(0x7f744c06c850 msgr2=0x7f744c119a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 shutdown_connections 2024-04-24T23:27:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.085+0000 7f7453d42640 1 -- 172.21.15.102:0/2901209184 wait complete. 2024-04-24T23:27:01.103 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:27:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.245+0000 7f89a2aae640 1 -- 172.21.15.102:0/147663333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c102530 msgr2=0x7f899c102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.245+0000 7f89a2aae640 1 --2- 172.21.15.102:0/147663333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c102530 0x7f899c102910 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8984009900 tx=0x7f898402a410 comp rx=0 tx=0).stop 2024-04-24T23:27:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.245+0000 7f89a2aae640 1 -- 172.21.15.102:0/147663333 shutdown_connections 2024-04-24T23:27:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.245+0000 7f89a2aae640 1 --2- 172.21.15.102:0/147663333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c100a10 0x7f899c110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.245+0000 7f89a2aae640 1 --2- 172.21.15.102:0/147663333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f899c100070 0x7f899c1004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.245+0000 7f89a2aae640 1 --2- 172.21.15.102:0/147663333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c102530 0x7f899c102910 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.245+0000 7f89a2aae640 1 -- 172.21.15.102:0/147663333 >> 172.21.15.102:0/147663333 conn(0x7f899c0758d0 msgr2=0x7f899c075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.245+0000 7f89a2aae640 1 -- 172.21.15.102:0/147663333 shutdown_connections 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.245+0000 7f89a2aae640 1 -- 172.21.15.102:0/147663333 wait complete. 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a2aae640 1 Processor -- start 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a2aae640 1 -- start start 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a2aae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c100070 0x7f899c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a2aae640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c100a10 0x7f899c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a2aae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f899c102530 0x7f899c19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a2aae640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f899c113cc0 con 0x7f899c100a10 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a2aae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f899c113b40 con 0x7f899c102530 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a2aae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f899c113e40 con 0x7f899c100070 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f899bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c100a10 0x7f899c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f899bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c100a10 0x7f899c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34004/0 (socket says 172.21.15.102:34004) 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f899bfff640 1 -- 172.21.15.102:0/1710786064 learned_addr learned my addr 172.21.15.102:0/1710786064 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a104b640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f899c102530 0x7f899c19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a084a640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c100070 0x7f899c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f899bfff640 1 -- 172.21.15.102:0/1710786064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c100070 msgr2=0x7f899c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f899bfff640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c100070 0x7f899c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f899bfff640 1 -- 172.21.15.102:0/1710786064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f899c102530 msgr2=0x7f899c19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f899bfff640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f899c102530 0x7f899c19ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f899bfff640 1 -- 172.21.15.102:0/1710786064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c19f150 con 0x7f899c100a10 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f89a104b640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f899c102530 0x7f899c19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.246+0000 7f899bfff640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c100a10 0x7f899c19a6a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9f0 tx=0x7f898c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.247+0000 7f8999ffb640 1 -- 172.21.15.102:0/1710786064 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00cd80 con 0x7f899c100a10 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.247+0000 7f8999ffb640 1 -- 172.21.15.102:0/1710786064 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f898c0108e0 con 0x7f899c100a10 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.247+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c19f440 con 0x7f899c100a10 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.247+0000 7f8999ffb640 1 -- 172.21.15.102:0/1710786064 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c018920 con 0x7f899c100a10 2024-04-24T23:27:01.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.247+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c1a6cd0 con 0x7f899c100a10 2024-04-24T23:27:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.248+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899c0689b0 con 0x7f899c100a10 2024-04-24T23:27:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.249+0000 7f8999ffb640 1 -- 172.21.15.102:0/1710786064 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f898c010a90 con 0x7f899c100a10 2024-04-24T23:27:01.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.250+0000 7f8999ffb640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f89880a2d80 0x7f89880a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.250+0000 7f89a084a640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f89880a2d80 0x7f89880a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.250+0000 7f8999ffb640 1 -- 172.21.15.102:0/1710786064 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7f898c0cf720 con 0x7f899c100a10 2024-04-24T23:27:01.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.251+0000 7f89a084a640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f89880a2d80 0x7f89880a5240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8984031440 tx=0x7f8984005b50 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:01.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.261+0000 7f8999ffb640 1 -- 172.21.15.102:0/1710786064 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f898c09bc50 con 0x7f899c100a10 2024-04-24T23:27:01.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:01 smithi183 ceph-mon[33137]: pgmap v436: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-04-24T23:27:01.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:01 smithi183 ceph-mon[33137]: from='client.25765 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:01.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:01 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1042597902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:01.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:01 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2901209184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:01.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:01 smithi102 ceph-mon[29216]: pgmap v436: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-04-24T23:27:01.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:01 smithi102 ceph-mon[29216]: from='client.25765 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:01.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:01 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1042597902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:01.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:01 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2901209184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:01.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.586+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f899c10bed0 con 0x7f899c100a10 2024-04-24T23:27:01.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.588+0000 7f8999ffb640 1 -- 172.21.15.102:0/1710786064 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f898c09ff50 con 0x7f899c100a10 2024-04-24T23:27:01.589 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-24T23:27:01.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f89880a2d80 msgr2=0x7f89880a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f89880a2d80 0x7f89880a5240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8984031440 tx=0x7f8984005b50 comp rx=0 tx=0).stop 2024-04-24T23:27:01.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c100a10 msgr2=0x7f899c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c100a10 0x7f899c19a6a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9f0 tx=0x7f898c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 shutdown_connections 2024-04-24T23:27:01.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f899c102530 0x7f899c19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f89880a2d80 0x7f89880a5240 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f899c100a10 0x7f899c19a6a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 --2- 172.21.15.102:0/1710786064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c100070 0x7f899c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 >> 172.21.15.102:0/1710786064 conn(0x7f899c0758d0 msgr2=0x7f899c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:01.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 shutdown_connections 2024-04-24T23:27:01.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.591+0000 7f89a2aae640 1 -- 172.21.15.102:0/1710786064 wait complete. 2024-04-24T23:27:01.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:01 smithi161 ceph-mon[32709]: pgmap v436: 97 pgs: 97 active+clean; 19 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-04-24T23:27:01.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:01 smithi161 ceph-mon[32709]: from='client.25765 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:27:01.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:01 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1042597902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:27:01.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:01 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2901209184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:27:01.609 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:27:01.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.757+0000 7fe43ef63640 1 -- 172.21.15.102:0/2256293364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4381032e0 msgr2=0x7fe43810b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.757+0000 7fe43ef63640 1 --2- 172.21.15.102:0/2256293364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4381032e0 0x7fe43810b7c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe424009900 tx=0x7fe42402a520 comp rx=0 tx=0).stop 2024-04-24T23:27:01.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.758+0000 7fe43ef63640 1 -- 172.21.15.102:0/2256293364 shutdown_connections 2024-04-24T23:27:01.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.758+0000 7fe43ef63640 1 --2- 172.21.15.102:0/2256293364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43810bd00 0x7fe43810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.758+0000 7fe43ef63640 1 --2- 172.21.15.102:0/2256293364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4381032e0 0x7fe43810b7c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.758+0000 7fe43ef63640 1 --2- 172.21.15.102:0/2256293364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4381029c0 0x7fe438102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.758+0000 7fe43ef63640 1 -- 172.21.15.102:0/2256293364 >> 172.21.15.102:0/2256293364 conn(0x7fe438077740 msgr2=0x7fe438077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:01.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.758+0000 7fe43ef63640 1 -- 172.21.15.102:0/2256293364 shutdown_connections 2024-04-24T23:27:01.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.758+0000 7fe43ef63640 1 -- 172.21.15.102:0/2256293364 wait complete. 2024-04-24T23:27:01.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43ef63640 1 Processor -- start 2024-04-24T23:27:01.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43ef63640 1 -- start start 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43ef63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4381029c0 0x7fe438199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43ef63640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4381032e0 0x7fe43819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43ef63640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43810bd00 0x7fe43819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43ef63640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe4381119b0 con 0x7fe43810bd00 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43ef63640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe438111830 con 0x7fe4381029c0 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe437fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4381032e0 0x7fe43819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe437fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4381032e0 0x7fe43819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.102:40030/0 (socket says 172.21.15.102:40030) 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe437fff640 1 -- 172.21.15.102:0/3684399390 learned_addr learned my addr 172.21.15.102:0/3684399390 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43ccff640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4381029c0 0x7fe438199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43d500640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43810bd00 0x7fe43819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.759+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe438111b30 con 0x7fe4381032e0 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe437fff640 1 -- 172.21.15.102:0/3684399390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4381029c0 msgr2=0x7fe438199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe437fff640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4381029c0 0x7fe438199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe437fff640 1 -- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43810bd00 msgr2=0x7fe43819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:01.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe437fff640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43810bd00 0x7fe43819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:01.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe437fff640 1 -- 172.21.15.102:0/3684399390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43819ef20 con 0x7fe4381032e0 2024-04-24T23:27:01.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe437fff640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4381032e0 0x7fe43819a470 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe424032fd0 tx=0x7fe424030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:01.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe435ffb640 1 -- 172.21.15.102:0/3684399390 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42403f070 con 0x7fe4381032e0 2024-04-24T23:27:01.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43819f1b0 con 0x7fe4381032e0 2024-04-24T23:27:01.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe435ffb640 1 -- 172.21.15.102:0/3684399390 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe424004460 con 0x7fe4381032e0 2024-04-24T23:27:01.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe435ffb640 1 -- 172.21.15.102:0/3684399390 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424035030 con 0x7fe4381032e0 2024-04-24T23:27:01.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.760+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4381a6aa0 con 0x7fe4381032e0 2024-04-24T23:27:01.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.762+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe438068560 con 0x7fe4381032e0 2024-04-24T23:27:01.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.763+0000 7fe435ffb640 1 -- 172.21.15.102:0/3684399390 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fe424042070 con 0x7fe4381032e0 2024-04-24T23:27:01.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.763+0000 7fe435ffb640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe4280a2d80 0x7fe4280a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:01.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.764+0000 7fe435ffb640 1 -- 172.21.15.102:0/3684399390 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7fe4240efb00 con 0x7fe4381032e0 2024-04-24T23:27:01.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.764+0000 7fe43ccff640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe4280a2d80 0x7fe4280a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:01.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.765+0000 7fe43ccff640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe4280a2d80 0x7fe4280a5240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0095f0 tx=0x7fe42c005f90 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:01.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:01.772+0000 7fe435ffb640 1 -- 172.21.15.102:0/3684399390 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe4240bc680 con 0x7fe4381032e0 2024-04-24T23:27:02.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.064+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe4381092b0 con 0x7fe4381032e0 2024-04-24T23:27:02.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.065+0000 7fe435ffb640 1 -- 172.21.15.102:0/3684399390 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7fe4240bbf80 con 0x7fe4381032e0 2024-04-24T23:27:02.066 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-24T23:27:02.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.069+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe4280a2d80 msgr2=0x7fe4280a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.069+0000 7fe43ef63640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe4280a2d80 0x7fe4280a5240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0095f0 tx=0x7fe42c005f90 comp rx=0 tx=0).stop 2024-04-24T23:27:02.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.069+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4381032e0 msgr2=0x7fe43819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.069+0000 7fe43ef63640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4381032e0 0x7fe43819a470 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe424032fd0 tx=0x7fe424030650 comp rx=0 tx=0).stop 2024-04-24T23:27:02.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.070+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 shutdown_connections 2024-04-24T23:27:02.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.070+0000 7fe43ef63640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fe4280a2d80 0x7fe4280a5240 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.070+0000 7fe43ef63640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43810bd00 0x7fe43819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.070+0000 7fe43ef63640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4381032e0 0x7fe43819a470 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.070+0000 7fe43ef63640 1 --2- 172.21.15.102:0/3684399390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4381029c0 0x7fe438199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.070+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 >> 172.21.15.102:0/3684399390 conn(0x7fe438077740 msgr2=0x7fe4381018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:02.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.071+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 shutdown_connections 2024-04-24T23:27:02.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.071+0000 7fe43ef63640 1 -- 172.21.15.102:0/3684399390 wait complete. 2024-04-24T23:27:02.088 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-24T23:27:02.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.238+0000 7fbfbcb42640 1 -- 172.21.15.102:0/172817629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8107710 msgr2=0x7fbfb8107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.238+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/172817629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8107710 0x7fbfb8107af0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009930 tx=0x7fbfa802a420 comp rx=0 tx=0).stop 2024-04-24T23:27:02.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.238+0000 7fbfbcb42640 1 -- 172.21.15.102:0/172817629 shutdown_connections 2024-04-24T23:27:02.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.238+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/172817629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb81020c0 0x7fbfb810e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.238+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/172817629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb8101700 0x7fbfb8101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.238+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/172817629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8107710 0x7fbfb8107af0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.238+0000 7fbfbcb42640 1 -- 172.21.15.102:0/172817629 >> 172.21.15.102:0/172817629 conn(0x7fbfb8078b40 msgr2=0x7fbfb8078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:02.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.238+0000 7fbfbcb42640 1 -- 172.21.15.102:0/172817629 shutdown_connections 2024-04-24T23:27:02.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.238+0000 7fbfbcb42640 1 -- 172.21.15.102:0/172817629 wait complete. 2024-04-24T23:27:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.239+0000 7fbfbcb42640 1 Processor -- start 2024-04-24T23:27:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.239+0000 7fbfbcb42640 1 -- start start 2024-04-24T23:27:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.239+0000 7fbfbcb42640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8101700 0x7fbfb8197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.239+0000 7fbfbcb42640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb81020c0 0x7fbfb81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfbcb42640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb8107710 0x7fbfb819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8101700 0x7fbfb8197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfbcb42640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81119d0 con 0x7fbfb8101700 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8101700 0x7fbfb8197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34030/0 (socket says 172.21.15.102:34030) 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb8111850 con 0x7fbfb81020c0 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb8111b50 con 0x7fbfb8107710 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb659c640 1 -- 172.21.15.102:0/3142842555 learned_addr learned my addr 172.21.15.102:0/3142842555 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfaffff640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb81020c0 0x7fbfb81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb6d9d640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb8107710 0x7fbfb819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb659c640 1 -- 172.21.15.102:0/3142842555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb8107710 msgr2=0x7fbfb819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb659c640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb8107710 0x7fbfb819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb659c640 1 -- 172.21.15.102:0/3142842555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb81020c0 msgr2=0x7fbfb81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb659c640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb81020c0 0x7fbfb81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb659c640 1 -- 172.21.15.102:0/3142842555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb819cf60 con 0x7fbfb8101700 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfb659c640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8101700 0x7fbfb8197f70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005940 tx=0x7fbfa80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.240+0000 7fbfaf7fe640 1 -- 172.21.15.102:0/3142842555 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa803f030 con 0x7fbfb8101700 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.241+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb819d1f0 con 0x7fbfb8101700 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.241+0000 7fbfaf7fe640 1 -- 172.21.15.102:0/3142842555 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa80044c0 con 0x7fbfb8101700 2024-04-24T23:27:02.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.241+0000 7fbfaf7fe640 1 -- 172.21.15.102:0/3142842555 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8035070 con 0x7fbfb8101700 2024-04-24T23:27:02.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.241+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb81a4ae0 con 0x7fbfb8101700 2024-04-24T23:27:02.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.242+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb80689b0 con 0x7fbfb8101700 2024-04-24T23:27:02.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.243+0000 7fbfaf7fe640 1 -- 172.21.15.102:0/3142842555 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fbfa8042030 con 0x7fbfb8101700 2024-04-24T23:27:02.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.244+0000 7fbfaf7fe640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fbfa40a2d00 0x7fbfa40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:27:02.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.244+0000 7fbfaf7fe640 1 -- 172.21.15.102:0/3142842555 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8579+0+0 (secure 0 0 0) 0x7fbfa80ef920 con 0x7fbfb8101700 2024-04-24T23:27:02.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.245+0000 7fbfaffff640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fbfa40a2d00 0x7fbfa40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:27:02.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.245+0000 7fbfaffff640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fbfa40a2d00 0x7fbfa40a51c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8103a60 tx=0x7fbfa0005f90 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:27:02.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.249+0000 7fbfaf7fe640 1 -- 172.21.15.102:0/3142842555 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbfa80bbeb0 con 0x7fbfb8101700 2024-04-24T23:27:02.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:02 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:02.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:02 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1710786064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:27:02.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:02 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3684399390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:27:02.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:02 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:02.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:02 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1710786064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:27:02.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:02 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3684399390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:27:02.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:02 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:02.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:02 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1710786064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:27:02.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:02 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3684399390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:27:02.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.639+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fbfb8109c70 con 0x7fbfb8101700 2024-04-24T23:27:02.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.647+0000 7fbfaf7fe640 1 -- 172.21.15.102:0/3142842555 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740908 (secure 0 0 0) 0x7fbfa80c01b0 con 0x7fbfb8101700 2024-04-24T23:27:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:27:02.658 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:27:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "epoch": 26, 2024-04-24T23:27:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "active_gid": 15288, 2024-04-24T23:27:02.658 INFO:teuthology.orchestra.run.smithi102.stdout: "active_name": "x", 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "active_addrs": { 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v2", 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6800", 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 161836032 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-04-24T23:27:02.659 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:6801", 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 161836032 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "active_addr": "172.21.15.102:6801/161836032", 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "active_change": "2024-04-24T23:13:01.525817+0000", 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "active_mgr_features": 4540701547738038271, 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "available": true, 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "standbys": [ 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "gid": 24799, 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "z", 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr_features": 4540701547738038271, 2024-04-24T23:27:02.660 INFO:teuthology.orchestra.run.smithi102.stdout: "available_modules": [ 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "alerts", 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.661 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.662 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.663 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.664 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.665 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_destination": { 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_destination", 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_from_name": { 2024-04-24T23:27:02.667 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_from_name", 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "Ceph", 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Email From: name", 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_host": { 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_host", 2024-04-24T23:27:02.668 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SMTP server", 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_password": { 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_password", 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.669 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Password to authenticate with", 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_port": { 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_port", 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.670 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "465", 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SMTP port", 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_sender": { 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_sender", 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.671 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_ssl": { 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_ssl", 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.672 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_user": { 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_user", 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.673 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "User to authenticate as", 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "balancer", 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.674 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "active": { 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "active", 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.675 INFO:teuthology.orchestra.run.smithi102.stdout: "begin_time": { 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "begin_time", 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0000", 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "begin_weekday": { 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "begin_weekday", 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "0", 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "6", 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.677 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_compat_max_iterations": { 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "25", 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "250", 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_compat_metrics": { 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crush_compat_metrics", 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.679 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_compat_step": { 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crush_compat_step", 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0.5", 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "0.001", 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "0.999", 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.680 INFO:teuthology.orchestra.run.smithi102.stdout: "end_time": { 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "end_time", 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "2359", 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "end_weekday": { 2024-04-24T23:27:02.681 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "end_weekday", 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "0", 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "6", 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.682 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.683 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.684 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.685 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.686 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "min_score": { 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "min_score", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.687 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "mode": { 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mode", 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.688 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "upmap", 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "crush-compat", 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "none", 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "read", 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap", 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap-read" 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Balancer mode", 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_ids": { 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pool_ids", 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.690 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap_max_deviation": { 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "upmap_max_deviation", 2024-04-24T23:27:02.691 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap_max_optimizations": { 2024-04-24T23:27:02.692 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.693 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cephadm", 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "agent_down_multiplier": { 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "agent_down_multiplier", 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.694 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "3.0", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "agent_refresh_rate": { 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "agent_refresh_rate", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "20", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.695 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "agent_starting_port": { 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "agent_starting_port", 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "4721", 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "allow_ptrace": { 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "allow_ptrace", 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.697 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "autotune_interval": { 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "autotune_interval", 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.698 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0.7", 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.699 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm_log_destination": { 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cephadm_log_destination", 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.700 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "file", 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "file,syslog", 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "syslog" 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "cgroups_split": { 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cgroups_split", 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.701 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "config_checks_enabled": { 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "config_checks_enabled", 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.702 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "config_dashboard": { 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "config_dashboard", 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.703 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_alertmanager": { 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_alertmanager", 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.704 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_base": { 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_base", 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:27:02.705 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_elasticsearch": { 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:27:02.706 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "elasticsearch container image", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_grafana": { 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_grafana", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.707 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_haproxy": { 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_haproxy", 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.708 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "HAproxy container image", 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_jaeger_agent": { 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.709 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_jaeger_collector": { 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.710 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_jaeger_query": { 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Jaeger query container image", 2024-04-24T23:27:02.711 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_keepalived": { 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_keepalived", 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Keepalived container image", 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.712 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_loki": { 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_loki", 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Loki container image", 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.713 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_node_exporter": { 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_node_exporter", 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.714 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_nvmeof": { 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_nvmeof", 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Nvme-of container image", 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.715 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_prometheus": { 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_prometheus", 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:02.716 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_promtail": { 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_promtail", 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Promtail container image", 2024-04-24T23:27:02.717 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_samba": { 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_samba", 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.718 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_snmp_gateway": { 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.719 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "container_init": { 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_init", 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.720 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "daemon_cache_timeout": { 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:27:02.721 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "900", 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:27:02.722 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "default_registry": { 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "default_registry", 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io", 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.723 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "device_cache_timeout": { 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "device_cache_timeout", 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "1800", 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.724 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "device_enhanced_scan": { 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "device_enhanced_scan", 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.725 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "facts_cache_timeout": { 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "facts_cache_timeout", 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "grafana_dashboards_path": { 2024-04-24T23:27:02.726 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "host_check_interval": { 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "host_check_interval", 2024-04-24T23:27:02.727 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "hw_monitoring": { 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "hw_monitoring", 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.728 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "inventory_list_all": { 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "inventory_list_all", 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.729 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.730 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: "log_refresh_metadata": { 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_refresh_metadata", 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.732 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.733 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.734 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.735 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.736 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "*", 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.737 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "max_count_per_host": { 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_count_per_host", 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.738 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "max_osd_draining_count": { 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_osd_draining_count", 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.739 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "migration_current": { 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "migration_current", 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "internal - do not modify", 2024-04-24T23:27:02.740 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "mode": { 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mode", 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "root", 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm-package", 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "root" 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.741 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "oob_default_addr": { 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "oob_default_addr", 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "169.254.1.1", 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:27:02.742 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "prometheus_alerts_path": { 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.743 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_insecure": { 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_insecure", 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.744 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_password": { 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_password", 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.745 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_url": { 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_url", 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.746 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_username": { 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_username", 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.747 INFO:teuthology.orchestra.run.smithi102.stdout: "secure_monitoring_stack": { 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.748 INFO:teuthology.orchestra.run.smithi102.stdout: "service_discovery_port": { 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "service_discovery_port", 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8765", 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "ssh_config_file": { 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssh_config_file", 2024-04-24T23:27:02.749 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "use_agent": { 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "use_agent", 2024-04-24T23:27:02.750 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "use_repo_digest": { 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "use_repo_digest", 2024-04-24T23:27:02.751 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_on_failed_host_check": { 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.752 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_on_stray_daemons": { 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.753 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_on_stray_hosts": { 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.754 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crash", 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.755 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.756 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.757 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.758 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.759 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "retain_interval": { 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "retain_interval", 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.760 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "31536000", 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_recent_interval": { 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_recent_interval", 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.761 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "1209600", 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.762 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "dashboard", 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.764 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.765 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.766 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.767 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.768 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.769 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.770 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:27:02.771 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:27:02.772 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.773 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.775 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "admin", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.776 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.777 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_URL": { 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.778 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "admin", 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.779 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.780 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:27:02.783 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.784 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.785 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.786 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.787 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.788 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.789 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.790 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.791 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.792 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.793 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.794 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8", 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.795 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "45", 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.796 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:27:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "admin", 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:27:02.798 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.799 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.800 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:02.801 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:02.802 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "cross_origin_url": { 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cross_origin_url", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.803 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "crt_file": { 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crt_file", 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.804 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "debug": { 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "debug", 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.805 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "jwt_token_ttl": { 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "jwt_token_ttl", 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "28800", 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "key_file": { 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "key_file", 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.808 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.809 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.810 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.811 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: "motd": { 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "motd", 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "The message of the day", 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.813 INFO:teuthology.orchestra.run.smithi102.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.814 INFO:teuthology.orchestra.run.smithi102.stdout: "server_addr": { 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_addr", 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "::", 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.815 INFO:teuthology.orchestra.run.smithi102.stdout: "server_port": { 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_port", 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8080", 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "ssl": { 2024-04-24T23:27:02.816 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssl", 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "ssl_server_port": { 2024-04-24T23:27:02.817 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssl_server_port", 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8443", 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.818 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_behaviour": { 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_behaviour", 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "redirect", 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "redirect" 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.819 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_error_status_code": { 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_error_status_code", 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "500", 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "400", 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "599", 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.820 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "url_prefix": { 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "url_prefix", 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.821 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "devicehealth", 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "enable_monitoring": { 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enable_monitoring", 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.822 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "monitor device health metrics", 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.823 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.824 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.825 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.826 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.827 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "mark_out_threshold": { 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mark_out_threshold", 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "2419200", 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.828 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_name": { 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pool_name", 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "device_health_metrics", 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:27:02.829 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "retention_period": { 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "retention_period", 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "15552000", 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:27:02.830 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "scrape_frequency": { 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "scrape_frequency", 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "86400", 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:27:02.831 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "self_heal": { 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "self_heal", 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:27:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.833 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_threshold": { 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_threshold", 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "7257600", 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.834 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "diskprediction_local", 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.836 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.837 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.838 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.839 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "predict_interval": { 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "predict_interval", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "86400", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.840 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "predictor_model": { 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "predictor_model", 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "prophetstor", 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.841 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.842 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "influx", 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": false, 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "batch_size": { 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "batch_size", 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5000", 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.843 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "database": { 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "database", 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "ceph", 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.844 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "hostname": { 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "hostname", 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.845 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "30", 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "5", 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.846 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.847 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.848 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.849 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.850 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "password": { 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "password", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.851 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "port": { 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "port", 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8086", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "InfluxDB server port", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "ssl": { 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssl", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "false", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.853 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "threads": { 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "threads", 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "32", 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.854 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "username": { 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "username", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.855 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "verify_ssl": { 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "verify_ssl", 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "true", 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.856 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "insights", 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.857 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.858 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.859 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.860 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.861 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "iostat", 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.862 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.863 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.863 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.863 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.863 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.863 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.863 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.863 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.863 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.866 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.867 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.868 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.869 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.870 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "k8sevents", 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_event_retention_days": { 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "7", 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.871 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "config_check_secs": { 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "config_check_secs", 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "10", 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.872 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.873 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.876 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.877 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "localpool", 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "failure_domain": { 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "failure_domain", 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "host", 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.878 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.879 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.880 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.881 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.882 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.883 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "min_size": { 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "min_size", 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.884 INFO:teuthology.orchestra.run.smithi102.stdout: "num_rep": { 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "num_rep", 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "3", 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_num": { 2024-04-24T23:27:02.885 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pg_num", 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "128", 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "prefix": { 2024-04-24T23:27:02.886 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "prefix", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "subtree": { 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "subtree", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.887 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "rack", 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mds_autoscaler", 2024-04-24T23:27:02.888 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.889 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.890 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.891 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.892 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mirroring", 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.893 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.894 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.895 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.896 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.897 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.898 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "nfs", 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.899 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.900 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.901 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.902 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "orchestrator", 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "fail_fs": { 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "fail_fs", 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.904 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.905 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.906 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.907 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.908 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "orchestrator": { 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "orchestrator", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.909 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm", 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "rook", 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "test_orchestrator" 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Orchestrator backend", 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.910 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "osd_perf_query", 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.911 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.912 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.913 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.914 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.915 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "osd_support", 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.916 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.917 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.918 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.919 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.920 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pg_autoscaler", 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.921 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.922 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.923 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.924 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.925 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.926 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "threshold": { 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "threshold", 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.927 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "3.0", 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1.0", 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "scaling threshold", 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "progress", 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.928 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "allow_pg_recovery_event": { 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.929 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "enabled": { 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enabled", 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.930 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.931 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.932 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.933 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.934 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "max_completed_events": { 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_completed_events", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "50", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.935 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.936 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "prometheus", 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.937 INFO:teuthology.orchestra.run.smithi102.stdout: "cache": { 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cache", 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.938 INFO:teuthology.orchestra.run.smithi102.stdout: "exclude_perf_counters": { 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "exclude_perf_counters", 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.939 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.940 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.941 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.942 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.943 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.944 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_stats_pools": { 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_stats_pools", 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.945 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "300", 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "scrape_interval": { 2024-04-24T23:27:02.946 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "scrape_interval", 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "15.0", 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "server_addr": { 2024-04-24T23:27:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_addr", 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "::", 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "server_port": { 2024-04-24T23:27:02.948 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_port", 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "9283", 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "stale_cache_strategy": { 2024-04-24T23:27:02.949 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "stale_cache_strategy", 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "log", 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.950 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_behaviour": { 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_behaviour", 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "default", 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "default", 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "error" 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.951 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_error_status_code": { 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_error_status_code", 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "500", 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "400", 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "599", 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.952 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_support", 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.953 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.954 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.956 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.958 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.958 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.958 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.958 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.958 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.958 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.958 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.959 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.959 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.959 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.959 INFO:teuthology.orchestra.run.smithi102.stdout: "max_concurrent_snap_create": { 2024-04-24T23:27:02.959 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:27:02.959 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.959 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.959 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.960 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "trash_purge_schedule": { 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "trash_purge_schedule", 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.961 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "restful", 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "enable_auth": { 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enable_auth", 2024-04-24T23:27:02.962 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "key_file": { 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "key_file", 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.963 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.964 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.965 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.966 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.967 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.968 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "server_addr": { 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_addr", 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.969 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "server_port": { 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_port", 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.970 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rgw", 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.971 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.972 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.973 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.974 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.975 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rook", 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.976 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.977 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.978 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.979 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.980 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "storage_class": { 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "storage_class", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.981 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "local", 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "selftest", 2024-04-24T23:27:02.982 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.983 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.984 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:02.985 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.986 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "roption1": { 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "roption1", 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.987 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "roption2": { 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "roption2", 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "xyz", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption1": { 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption1", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption2": { 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption2", 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption3": { 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption3", 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.991 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption4": { 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption4", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.992 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption5": { 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption5", 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.993 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption6": { 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption6", 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.994 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption7": { 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption7", 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "42", 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.995 INFO:teuthology.orchestra.run.smithi102.stdout: "testkey": { 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "testkey", 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "testlkey": { 2024-04-24T23:27:02.996 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "testlkey", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "testnewline": { 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "testnewline", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:02.997 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:02.998 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snap_schedule", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "allow_m_granularity": { 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "allow_m_granularity", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:02.999 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "dump_on_update": { 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "dump_on_update", 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.000 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.001 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.002 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.003 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.004 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "stats", 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.005 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.006 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.007 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.008 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.009 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.010 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "status", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.011 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.012 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.013 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.014 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.015 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "telegraf", 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "address": { 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "address", 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:27:03.016 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "15", 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.017 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.018 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.019 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.020 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.021 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.022 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "telemetry", 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_basic": { 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_basic", 2024-04-24T23:27:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_crash": { 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_crash", 2024-04-24T23:27:03.024 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_device": { 2024-04-24T23:27:03.025 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_device", 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.026 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_ident": { 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_ident", 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_perf": { 2024-04-24T23:27:03.027 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_perf", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "contact": { 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "contact", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.028 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "description": { 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "description", 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.029 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "device_url": { 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "device_url", 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.030 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "enabled": { 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enabled", 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.031 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "24", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "8", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.032 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "last_opt_revision": { 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "last_opt_revision", 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "1", 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.033 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "leaderboard": { 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "leaderboard", 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.034 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "leaderboard_description": { 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "leaderboard_description", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.035 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.036 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.037 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.038 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.039 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.040 INFO:teuthology.orchestra.run.smithi102.stdout: "organization": { 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "organization", 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.041 INFO:teuthology.orchestra.run.smithi102.stdout: "proxy": { 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "proxy", 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.042 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "url": { 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "url", 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.043 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "test_orchestrator", 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.044 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.045 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.046 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.047 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "volumes", 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.049 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.050 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.051 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.052 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.053 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "max_concurrent_clones": { 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_concurrent_clones", 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.054 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "4", 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "periodic_async_work": { 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "periodic_async_work", 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.055 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Periodically check for async work", 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "snapshot_clone_delay": { 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.056 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.057 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix", 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.058 INFO:teuthology.orchestra.run.smithi102.stdout: "discovery_interval": { 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "discovery_interval", 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "100", 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "identifier": { 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "identifier", 2024-04-24T23:27:03.059 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.060 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.061 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.062 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.063 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "zabbix_host": { 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix_host", 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "zabbix_port": { 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix_port", 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10051", 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "zabbix_sender": { 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix_sender", 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.069 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.070 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "gid": 24803, 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "y", 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr_features": 4540701547738038271, 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "available_modules": [ 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "alerts", 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.071 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.072 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.075 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.076 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_destination": { 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_destination", 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.077 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_from_name": { 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_from_name", 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "Ceph", 2024-04-24T23:27:03.078 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Email From: name", 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_host": { 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_host", 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.079 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SMTP server", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_password": { 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_password", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.080 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Password to authenticate with", 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_port": { 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_port", 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "465", 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.081 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SMTP port", 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_sender": { 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_sender", 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.082 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_ssl": { 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_ssl", 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.083 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_user": { 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_user", 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "User to authenticate as", 2024-04-24T23:27:03.084 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "balancer", 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "active": { 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "active", 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.085 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "begin_time": { 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "begin_time", 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.086 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0000", 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "begin_weekday": { 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "begin_weekday", 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.087 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "0", 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "6", 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_compat_max_iterations": { 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.088 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "25", 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "250", 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_compat_metrics": { 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crush_compat_metrics", 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:27:03.089 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_compat_step": { 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crush_compat_step", 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0.5", 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "0.001", 2024-04-24T23:27:03.090 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "0.999", 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "end_time": { 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "end_time", 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "2359", 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.091 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "end_weekday": { 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "end_weekday", 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "0", 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "6", 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:27:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.093 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.094 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.095 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.096 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "min_score": { 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "min_score", 2024-04-24T23:27:03.097 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "mode": { 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mode", 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.098 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "upmap", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "crush-compat", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "none", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "read", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap-read" 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Balancer mode", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_ids": { 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pool_ids", 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.100 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap_max_deviation": { 2024-04-24T23:27:03.101 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "upmap_max_deviation", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap_max_optimizations": { 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.102 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cephadm", 2024-04-24T23:27:03.103 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "agent_down_multiplier": { 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "agent_down_multiplier", 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "3.0", 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.104 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "agent_refresh_rate": { 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "agent_refresh_rate", 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "20", 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "agent_starting_port": { 2024-04-24T23:27:03.105 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "agent_starting_port", 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "4721", 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "allow_ptrace": { 2024-04-24T23:27:03.106 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "allow_ptrace", 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "autotune_interval": { 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "autotune_interval", 2024-04-24T23:27:03.107 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.108 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0.7", 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm_log_destination": { 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cephadm_log_destination", 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.109 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "file", 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "file,syslog", 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "syslog" 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "cgroups_split": { 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cgroups_split", 2024-04-24T23:27:03.110 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "config_checks_enabled": { 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "config_checks_enabled", 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.111 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "config_dashboard": { 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "config_dashboard", 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.112 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_alertmanager": { 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_alertmanager", 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.113 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_base": { 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_base", 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:27:03.114 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_elasticsearch": { 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.115 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "elasticsearch container image", 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_grafana": { 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_grafana", 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.116 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_haproxy": { 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_haproxy", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "HAproxy container image", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.117 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_jaeger_agent": { 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.118 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_jaeger_collector": { 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:27:03.119 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_jaeger_query": { 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Jaeger query container image", 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.120 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_keepalived": { 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_keepalived", 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Keepalived container image", 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.121 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_loki": { 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_loki", 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Loki container image", 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_node_exporter": { 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_node_exporter", 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.123 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_nvmeof": { 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_nvmeof", 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Nvme-of container image", 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_prometheus": { 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_prometheus", 2024-04-24T23:27:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_promtail": { 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_promtail", 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.125 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Promtail container image", 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_samba": { 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_samba", 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.126 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_snmp_gateway": { 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:27:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "container_init": { 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_init", 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.128 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "daemon_cache_timeout": { 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.129 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "900", 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.130 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "default_registry": { 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "default_registry", 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io", 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:27:03.131 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "device_cache_timeout": { 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "device_cache_timeout", 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "1800", 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.132 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "device_enhanced_scan": { 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "device_enhanced_scan", 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.133 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "facts_cache_timeout": { 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "facts_cache_timeout", 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: "grafana_dashboards_path": { 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.135 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:27:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: "host_check_interval": { 2024-04-24T23:27:03.136 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "host_check_interval", 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.137 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "hw_monitoring": { 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "hw_monitoring", 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:27:03.138 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "inventory_list_all": { 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "inventory_list_all", 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:27:03.139 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.140 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "log_refresh_metadata": { 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_refresh_metadata", 2024-04-24T23:27:03.141 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.142 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.143 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.144 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.145 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:27:03.146 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "*", 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:27:03.147 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "max_count_per_host": { 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_count_per_host", 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:27:03.148 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "max_osd_draining_count": { 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_osd_draining_count", 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:27:03.149 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "migration_current": { 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "migration_current", 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "internal - do not modify", 2024-04-24T23:27:03.150 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "mode": { 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mode", 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "root", 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm-package", 2024-04-24T23:27:03.151 INFO:teuthology.orchestra.run.smithi102.stdout: "root" 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "oob_default_addr": { 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "oob_default_addr", 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "169.254.1.1", 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.152 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "prometheus_alerts_path": { 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.153 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_insecure": { 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_insecure", 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.154 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_password": { 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_password", 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.155 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_url": { 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_url", 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.156 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_username": { 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_username", 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.157 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "secure_monitoring_stack": { 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:27:03.158 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "service_discovery_port": { 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "service_discovery_port", 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8765", 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.159 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "ssh_config_file": { 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssh_config_file", 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.160 INFO:teuthology.orchestra.run.smithi102.stdout: "use_agent": { 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "use_agent", 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "use_repo_digest": { 2024-04-24T23:27:03.161 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "use_repo_digest", 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_on_failed_host_check": { 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:27:03.162 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_on_stray_daemons": { 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.163 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_on_stray_hosts": { 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.164 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crash", 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.166 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.167 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.168 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "retain_interval": { 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "retain_interval", 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "31536000", 2024-04-24T23:27:03.170 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_recent_interval": { 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_recent_interval", 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "1209600", 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "dashboard", 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:27:03.172 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.173 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.174 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.175 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.176 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.178 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.179 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "admin", 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_URL": { 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "admin", 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:27:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:27:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:27:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8", 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "45", 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "admin", 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:27:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:27:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "cross_origin_url": { 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cross_origin_url", 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "crt_file": { 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crt_file", 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "debug": { 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "debug", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "jwt_token_ttl": { 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "jwt_token_ttl", 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "28800", 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "key_file": { 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "key_file", 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "motd": { 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "motd", 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "The message of the day", 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "server_addr": { 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_addr", 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "::", 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "server_port": { 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_port", 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8080", 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "ssl": { 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssl", 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.228 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.228 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.228 INFO:teuthology.orchestra.run.smithi102.stdout: "ssl_server_port": { 2024-04-24T23:27:03.228 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssl_server_port", 2024-04-24T23:27:03.228 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.228 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.228 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.228 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8443", 2024-04-24T23:27:03.229 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.229 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.229 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.229 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.229 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.229 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.229 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.229 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_behaviour": { 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_behaviour", 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "redirect", 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.230 INFO:teuthology.orchestra.run.smithi102.stdout: "redirect" 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_error_status_code": { 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_error_status_code", 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "500", 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "400", 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "599", 2024-04-24T23:27:03.231 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "url_prefix": { 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "url_prefix", 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.232 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "devicehealth", 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "enable_monitoring": { 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enable_monitoring", 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.233 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "monitor device health metrics", 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.234 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.235 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.236 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.237 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.238 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "mark_out_threshold": { 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mark_out_threshold", 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "2419200", 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.239 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_name": { 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pool_name", 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "device_health_metrics", 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.240 INFO:teuthology.orchestra.run.smithi102.stdout: "retention_period": { 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "retention_period", 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "15552000", 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.241 INFO:teuthology.orchestra.run.smithi102.stdout: "scrape_frequency": { 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "scrape_frequency", 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "86400", 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.242 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "self_heal": { 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "self_heal", 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.243 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.244 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_threshold": { 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_threshold", 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "7257600", 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.245 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "diskprediction_local", 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.246 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.247 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.248 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.249 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.250 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "predict_interval": { 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "predict_interval", 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "86400", 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.251 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "predictor_model": { 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "predictor_model", 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "prophetstor", 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.252 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "influx", 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": false, 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "batch_size": { 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "batch_size", 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.254 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5000", 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "database": { 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "database", 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.255 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "ceph", 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "hostname": { 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "hostname", 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.256 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.257 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "30", 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "5", 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.258 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.259 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.260 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.261 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.262 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "password": { 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "password", 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.263 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "port": { 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "port", 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8086", 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "InfluxDB server port", 2024-04-24T23:27:03.264 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "ssl": { 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssl", 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "false", 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.265 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "threads": { 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "threads", 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "32", 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "username": { 2024-04-24T23:27:03.266 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "username", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "verify_ssl": { 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "verify_ssl", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.267 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "true", 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "insights", 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.270 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.271 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.272 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "iostat", 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.273 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.275 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.276 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.277 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "k8sevents", 2024-04-24T23:27:03.278 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_event_retention_days": { 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "7", 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.279 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "config_check_secs": { 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "config_check_secs", 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "10", 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.280 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.281 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.282 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.283 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.284 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "localpool", 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "failure_domain": { 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "failure_domain", 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "host", 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.286 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.287 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.289 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.290 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "min_size": { 2024-04-24T23:27:03.291 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "min_size", 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "num_rep": { 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "num_rep", 2024-04-24T23:27:03.292 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "3", 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_num": { 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pg_num", 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.293 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "128", 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.294 INFO:teuthology.orchestra.run.smithi102.stdout: "prefix": { 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "prefix", 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "subtree": { 2024-04-24T23:27:03.295 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "subtree", 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "rack", 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.296 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mds_autoscaler", 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.297 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.299 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.300 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.301 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mirroring", 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.302 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.304 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.305 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.306 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "nfs", 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.307 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.308 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.309 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.310 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.311 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "orchestrator", 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "fail_fs": { 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "fail_fs", 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.312 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.313 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.314 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.315 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.316 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.317 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "orchestrator": { 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "orchestrator", 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm", 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "rook", 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "test_orchestrator" 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.318 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Orchestrator backend", 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "osd_perf_query", 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.319 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.320 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.321 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.322 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.323 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "osd_support", 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.324 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.325 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.326 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.327 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.328 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pg_autoscaler", 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.330 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.331 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.332 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.333 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.334 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "threshold": { 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "threshold", 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.335 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "3.0", 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1.0", 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "scaling threshold", 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "progress", 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "allow_pg_recovery_event": { 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.337 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "enabled": { 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enabled", 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.338 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.339 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.340 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.341 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.342 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.343 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "max_completed_events": { 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_completed_events", 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "50", 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.344 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.345 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "prometheus", 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "cache": { 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cache", 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "exclude_perf_counters": { 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "exclude_perf_counters", 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.347 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.348 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.349 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.350 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.351 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.352 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_stats_pools": { 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_stats_pools", 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.353 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "300", 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.354 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "scrape_interval": { 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "scrape_interval", 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "15.0", 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.355 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "server_addr": { 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_addr", 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "::", 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.356 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "server_port": { 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_port", 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "9283", 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.357 INFO:teuthology.orchestra.run.smithi102.stdout: "stale_cache_strategy": { 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "stale_cache_strategy", 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "log", 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_behaviour": { 2024-04-24T23:27:03.358 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_behaviour", 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "default", 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "default", 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "error" 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.359 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_error_status_code": { 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_error_status_code", 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "500", 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "400", 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "599", 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.360 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_support", 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.362 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.363 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.364 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.365 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "max_concurrent_snap_create": { 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.366 INFO:teuthology.orchestra.run.smithi102.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "trash_purge_schedule": { 2024-04-24T23:27:03.367 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "trash_purge_schedule", 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.368 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "restful", 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "enable_auth": { 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enable_auth", 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.369 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "key_file": { 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "key_file", 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.370 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.371 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "server_addr": { 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_addr", 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "server_port": { 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_port", 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rgw", 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.382 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rook", 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.383 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.384 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.385 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.386 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.387 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "storage_class": { 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "storage_class", 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "local", 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:27:03.388 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "selftest", 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.390 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.392 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.393 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "roption1": { 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "roption1", 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.394 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "roption2": { 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "roption2", 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "xyz", 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.395 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption1": { 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption1", 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.396 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption2": { 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption2", 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.397 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption3": { 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption3", 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.398 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption4": { 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption4", 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.399 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption5": { 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption5", 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.400 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption6": { 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption6", 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.401 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption7": { 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption7", 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "42", 2024-04-24T23:27:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "testkey": { 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "testkey", 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.403 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "testlkey": { 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "testlkey", 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.404 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "testnewline": { 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "testnewline", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.405 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snap_schedule", 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "allow_m_granularity": { 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "allow_m_granularity", 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.406 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "dump_on_update": { 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "dump_on_update", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.407 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.408 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.409 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.410 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "stats", 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.413 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.414 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.415 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.416 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "status", 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.419 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.420 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.421 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.422 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "telegraf", 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "address": { 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "address", 2024-04-24T23:27:03.423 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.424 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "15", 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.425 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.426 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.427 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.428 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.429 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "telemetry", 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.430 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_basic": { 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_basic", 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.431 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_crash": { 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_crash", 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.432 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_device": { 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_device", 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_ident": { 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_ident", 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_perf": { 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_perf", 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.435 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "contact": { 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "contact", 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.436 INFO:teuthology.orchestra.run.smithi102.stdout: "description": { 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "description", 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "device_url": { 2024-04-24T23:27:03.437 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "device_url", 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "enabled": { 2024-04-24T23:27:03.438 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enabled", 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.439 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "24", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "8", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "last_opt_revision": { 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "last_opt_revision", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.440 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "1", 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "leaderboard": { 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "leaderboard", 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.441 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "leaderboard_description": { 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "leaderboard_description", 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.442 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.443 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.444 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.445 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.446 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.447 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "organization": { 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "organization", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.448 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "proxy": { 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "proxy", 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.449 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "url": { 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "url", 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.450 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "test_orchestrator", 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.451 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.452 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.453 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.454 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.455 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "volumes", 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.456 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.457 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.459 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.460 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "max_concurrent_clones": { 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_concurrent_clones", 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "4", 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.461 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "periodic_async_work": { 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "periodic_async_work", 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.462 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Periodically check for async work", 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "snapshot_clone_delay": { 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:27:03.463 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:27:03.464 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix", 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "discovery_interval": { 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "discovery_interval", 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.465 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "100", 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "identifier": { 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "identifier", 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.466 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.467 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.468 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.469 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.470 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.471 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.472 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "zabbix_host": { 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix_host", 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.473 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "zabbix_port": { 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix_port", 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10051", 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.474 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "zabbix_sender": { 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix_sender", 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.475 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: "modules": [ 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm", 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: "dashboard", 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: "iostat", 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: "nfs", 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: "restful", 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: "snap_schedule" 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: "available_modules": [ 2024-04-24T23:27:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "alerts", 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.477 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.478 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.479 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.480 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.481 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.482 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_destination": { 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_destination", 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.483 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_from_name": { 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_from_name", 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "Ceph", 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Email From: name", 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_host": { 2024-04-24T23:27:03.484 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_host", 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SMTP server", 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_password": { 2024-04-24T23:27:03.485 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_password", 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Password to authenticate with", 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_port": { 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_port", 2024-04-24T23:27:03.486 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "465", 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SMTP port", 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_sender": { 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_sender", 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.487 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_ssl": { 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_ssl", 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.488 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "smtp_user": { 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "smtp_user", 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.489 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "User to authenticate as", 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "balancer", 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.490 INFO:teuthology.orchestra.run.smithi102.stdout: "active": { 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "active", 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "begin_time": { 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "begin_time", 2024-04-24T23:27:03.491 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0000", 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "begin_weekday": { 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "begin_weekday", 2024-04-24T23:27:03.492 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "0", 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "6", 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_compat_max_iterations": { 2024-04-24T23:27:03.493 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "25", 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "250", 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_compat_metrics": { 2024-04-24T23:27:03.494 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crush_compat_metrics", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "crush_compat_step": { 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crush_compat_step", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.495 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0.5", 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "0.001", 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "0.999", 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "end_time": { 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "end_time", 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.496 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "2359", 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "end_weekday": { 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "end_weekday", 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.497 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "0", 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "6", 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.498 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.499 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.500 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.501 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.502 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "min_score": { 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "min_score", 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.503 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "mode": { 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mode", 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "upmap", 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "crush-compat", 2024-04-24T23:27:03.504 INFO:teuthology.orchestra.run.smithi102.stdout: "none", 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "read", 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap", 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap-read" 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Balancer mode", 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_ids": { 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pool_ids", 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.505 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.506 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap_max_deviation": { 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "upmap_max_deviation", 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.507 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "upmap_max_optimizations": { 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.508 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cephadm", 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.509 INFO:teuthology.orchestra.run.smithi102.stdout: "agent_down_multiplier": { 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "agent_down_multiplier", 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "3.0", 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.510 INFO:teuthology.orchestra.run.smithi102.stdout: "agent_refresh_rate": { 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "agent_refresh_rate", 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "20", 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "agent_starting_port": { 2024-04-24T23:27:03.511 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "agent_starting_port", 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "4721", 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "allow_ptrace": { 2024-04-24T23:27:03.512 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "allow_ptrace", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "autotune_interval": { 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "autotune_interval", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.513 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.514 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0.7", 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm_log_destination": { 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cephadm_log_destination", 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.515 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "file", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "file,syslog", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "syslog" 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "cgroups_split": { 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cgroups_split", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.516 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "config_checks_enabled": { 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "config_checks_enabled", 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.517 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "config_dashboard": { 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "config_dashboard", 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.518 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_alertmanager": { 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_alertmanager", 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.519 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_base": { 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_base", 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:27:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_elasticsearch": { 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.521 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "elasticsearch container image", 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_grafana": { 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_grafana", 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.522 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_haproxy": { 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_haproxy", 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.523 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "HAproxy container image", 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_jaeger_agent": { 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:27:03.524 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_jaeger_collector": { 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.525 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_jaeger_query": { 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Jaeger query container image", 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.526 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_keepalived": { 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_keepalived", 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Keepalived container image", 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.527 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_loki": { 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_loki", 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Loki container image", 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.528 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_node_exporter": { 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_node_exporter", 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.529 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_nvmeof": { 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_nvmeof", 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Nvme-of container image", 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.530 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_prometheus": { 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_prometheus", 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Prometheus container image", 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.531 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_promtail": { 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_promtail", 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Promtail container image", 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_samba": { 2024-04-24T23:27:03.532 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_samba", 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "container_image_snmp_gateway": { 2024-04-24T23:27:03.533 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "container_init": { 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "container_init", 2024-04-24T23:27:03.534 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "daemon_cache_timeout": { 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "900", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.536 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "default_registry": { 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "default_registry", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "docker.io", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.537 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "device_cache_timeout": { 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "device_cache_timeout", 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "1800", 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.538 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "device_enhanced_scan": { 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "device_enhanced_scan", 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "facts_cache_timeout": { 2024-04-24T23:27:03.539 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "facts_cache_timeout", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "grafana_dashboards_path": { 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.540 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "host_check_interval": { 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "host_check_interval", 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.541 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "hw_monitoring": { 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "hw_monitoring", 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.542 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "inventory_list_all": { 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "inventory_list_all", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.543 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.544 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "log_refresh_metadata": { 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_refresh_metadata", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.545 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.546 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.547 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.548 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.549 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "*", 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.551 INFO:teuthology.orchestra.run.smithi102.stdout: "max_count_per_host": { 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_count_per_host", 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.552 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "max_osd_draining_count": { 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_osd_draining_count", 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.553 INFO:teuthology.orchestra.run.smithi102.stdout: "migration_current": { 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "migration_current", 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "internal - do not modify", 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "mode": { 2024-04-24T23:27:03.554 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mode", 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "root", 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm-package", 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "root" 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.555 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "oob_default_addr": { 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "oob_default_addr", 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "169.254.1.1", 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.556 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "prometheus_alerts_path": { 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.557 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_insecure": { 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_insecure", 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_password": { 2024-04-24T23:27:03.558 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_password", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_url": { 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_url", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.559 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "registry_username": { 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "registry_username", 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.560 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "secure_monitoring_stack": { 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.561 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "service_discovery_port": { 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "service_discovery_port", 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.562 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8765", 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "ssh_config_file": { 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssh_config_file", 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.563 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "use_agent": { 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "use_agent", 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.564 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "use_repo_digest": { 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "use_repo_digest", 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.565 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_on_failed_host_check": { 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.566 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_on_stray_daemons": { 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.567 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_on_stray_hosts": { 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.568 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crash", 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.569 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.570 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.571 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.573 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "retain_interval": { 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "retain_interval", 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "31536000", 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.574 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_recent_interval": { 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_recent_interval", 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "1209600", 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.575 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "dashboard", 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.576 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.577 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.578 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.579 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.581 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.582 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.584 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.585 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.586 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.587 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.588 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.589 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "admin", 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.590 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_URL": { 2024-04-24T23:27:03.591 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:27:03.592 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "admin", 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:27:03.593 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:27:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.595 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.596 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.597 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.598 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.600 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:27:03.601 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:27:03.602 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:27:03.603 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.604 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.605 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.606 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.607 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8", 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.608 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "45", 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.609 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.610 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "admin", 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.611 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:27:03.612 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:27:03.613 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:27:03.614 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:27:03.615 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "cross_origin_url": { 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cross_origin_url", 2024-04-24T23:27:03.616 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "crt_file": { 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "crt_file", 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.617 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "debug": { 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "debug", 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.618 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "jwt_token_ttl": { 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "jwt_token_ttl", 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.619 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "28800", 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "key_file": { 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "key_file", 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.620 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.621 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.622 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.623 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.624 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.625 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "motd": { 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "motd", 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.626 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "The message of the day", 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.627 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "server_addr": { 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_addr", 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "::", 2024-04-24T23:27:03.628 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "server_port": { 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_port", 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8080", 2024-04-24T23:27:03.629 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "ssl": { 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssl", 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.630 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "ssl_server_port": { 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssl_server_port", 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8443", 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.631 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_behaviour": { 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_behaviour", 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "redirect", 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.632 INFO:teuthology.orchestra.run.smithi102.stdout: "redirect" 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_error_status_code": { 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_error_status_code", 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "500", 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "400", 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "599", 2024-04-24T23:27:03.633 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "url_prefix": { 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "url_prefix", 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.634 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "devicehealth", 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "enable_monitoring": { 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enable_monitoring", 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.635 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "monitor device health metrics", 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.636 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.637 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.638 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.639 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.640 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "mark_out_threshold": { 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mark_out_threshold", 2024-04-24T23:27:03.641 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "2419200", 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "pool_name": { 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pool_name", 2024-04-24T23:27:03.642 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "device_health_metrics", 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "retention_period": { 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "retention_period", 2024-04-24T23:27:03.643 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "15552000", 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "scrape_frequency": { 2024-04-24T23:27:03.644 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "scrape_frequency", 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "86400", 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "self_heal": { 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "self_heal", 2024-04-24T23:27:03.645 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.646 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "warn_threshold": { 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "warn_threshold", 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.647 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "7257600", 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "diskprediction_local", 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.648 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.649 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.650 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.651 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.652 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "predict_interval": { 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "predict_interval", 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.653 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "86400", 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "predictor_model": { 2024-04-24T23:27:03.654 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "predictor_model", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "prophetstor", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.655 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "600", 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "influx", 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": false, 2024-04-24T23:27:03.656 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "batch_size": { 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "batch_size", 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5000", 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "database": { 2024-04-24T23:27:03.657 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "database", 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "ceph", 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "hostname": { 2024-04-24T23:27:03.658 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "hostname", 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.659 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "30", 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "5", 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.660 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.661 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.662 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.663 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.664 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "password": { 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "password", 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.665 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "port": { 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "port", 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "8086", 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.666 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "InfluxDB server port", 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "ssl": { 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ssl", 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "false", 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.667 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "threads": { 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "threads", 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "32", 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.668 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "username": { 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "username", 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.669 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "verify_ssl": { 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "verify_ssl", 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "true", 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "insights", 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.671 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.672 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.673 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.674 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.675 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "iostat", 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.676 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.677 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.678 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.679 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.679 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.679 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.679 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.679 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.681 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.681 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.681 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.681 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.681 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.681 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.682 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.683 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "k8sevents", 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_event_retention_days": { 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.684 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "7", 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "config_check_secs": { 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "config_check_secs", 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.685 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "10", 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.686 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.688 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.689 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.690 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "localpool", 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "failure_domain": { 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "failure_domain", 2024-04-24T23:27:03.691 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "host", 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.692 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.693 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.694 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.695 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.696 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "min_size": { 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "min_size", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "num_rep": { 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "num_rep", 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "3", 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.698 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_num": { 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pg_num", 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "128", 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.699 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "prefix": { 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "prefix", 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:27:03.700 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "subtree": { 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "subtree", 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "rack", 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:27:03.701 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mds_autoscaler", 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.702 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.703 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.704 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.706 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mirroring", 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.707 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.708 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.709 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.710 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.711 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "nfs", 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.712 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.713 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.714 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.715 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.716 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "orchestrator", 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.717 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "fail_fs": { 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "fail_fs", 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.718 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.719 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.720 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.721 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.722 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "orchestrator": { 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "orchestrator", 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.723 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "cephadm", 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "rook", 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "test_orchestrator" 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Orchestrator backend", 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.724 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "osd_perf_query", 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.725 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.726 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.727 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.728 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.729 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "osd_support", 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.730 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.731 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.732 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.733 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.734 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "pg_autoscaler", 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.735 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.736 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.737 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.738 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.739 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.740 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "threshold": { 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "threshold", 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "3.0", 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1.0", 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.741 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "scaling threshold", 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "progress", 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "allow_pg_recovery_event": { 2024-04-24T23:27:03.742 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "enabled": { 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enabled", 2024-04-24T23:27:03.743 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.744 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.745 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.746 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.748 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "max_completed_events": { 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_completed_events", 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "50", 2024-04-24T23:27:03.749 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "sleep_interval": { 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "sleep_interval", 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "5", 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.750 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "prometheus", 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.751 INFO:teuthology.orchestra.run.smithi102.stdout: "cache": { 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "cache", 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.752 INFO:teuthology.orchestra.run.smithi102.stdout: "exclude_perf_counters": { 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "exclude_perf_counters", 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.753 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.754 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.755 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.756 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.757 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_stats_pools": { 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_stats_pools", 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.758 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.759 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "300", 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "scrape_interval": { 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "scrape_interval", 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.760 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "15.0", 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "server_addr": { 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_addr", 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.761 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "::", 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "server_port": { 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_port", 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.762 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.763 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "9283", 2024-04-24T23:27:03.763 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.763 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.763 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.763 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:27:03.763 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.763 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.763 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.763 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "stale_cache_strategy": { 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "stale_cache_strategy", 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "log", 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.764 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_behaviour": { 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_behaviour", 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "default", 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "default", 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "error" 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.765 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "standby_error_status_code": { 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "standby_error_status_code", 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "500", 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "400", 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "599", 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.766 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_support", 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.767 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.768 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.769 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.770 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.771 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "max_concurrent_snap_create": { 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10", 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.772 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.773 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "trash_purge_schedule": { 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "trash_purge_schedule", 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.774 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "restful", 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "enable_auth": { 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enable_auth", 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "key_file": { 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "key_file", 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.776 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.778 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.779 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.780 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.781 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "server_addr": { 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_addr", 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.782 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "server_port": { 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "server_port", 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.783 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rgw", 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.785 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.786 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.787 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.788 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rook", 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.789 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.790 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.791 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.792 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.793 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "storage_class": { 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "storage_class", 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "local", 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.794 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "selftest", 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.795 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.796 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.797 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.798 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.799 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "roption1": { 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "roption1", 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.800 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "roption2": { 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "roption2", 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "xyz", 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.801 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption1": { 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption1", 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.802 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption2": { 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption2", 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.803 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption3": { 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption3", 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "float", 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.804 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption4": { 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption4", 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.805 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption5": { 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption5", 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.806 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption6": { 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption6", 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.807 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "rwoption7": { 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rwoption7", 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "1", 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "42", 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.808 INFO:teuthology.orchestra.run.smithi102.stdout: "testkey": { 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "testkey", 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "testlkey": { 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "testlkey", 2024-04-24T23:27:03.809 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "testnewline": { 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "testnewline", 2024-04-24T23:27:03.810 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.811 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snap_schedule", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "allow_m_granularity": { 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "allow_m_granularity", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.812 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "dump_on_update": { 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "dump_on_update", 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.813 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.814 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.815 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.816 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.817 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.818 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "stats", 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.819 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.820 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.821 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.822 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.823 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "status", 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.824 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.825 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.826 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.827 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.828 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "telegraf", 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "address": { 2024-04-24T23:27:03.829 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "address", 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.830 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "15", 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.831 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.832 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.833 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.834 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.835 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "telemetry", 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.836 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_basic": { 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_basic", 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.837 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_crash": { 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_crash", 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.838 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_device": { 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_device", 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.839 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_ident": { 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_ident", 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "channel_perf": { 2024-04-24T23:27:03.840 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "channel_perf", 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "contact": { 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "contact", 2024-04-24T23:27:03.841 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "description": { 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "description", 2024-04-24T23:27:03.842 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "device_url": { 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "device_url", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.843 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "enabled": { 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "enabled", 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.844 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.845 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "24", 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "8", 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "last_opt_revision": { 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "last_opt_revision", 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "1", 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "leaderboard": { 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "leaderboard", 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.847 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "leaderboard_description": { 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "leaderboard_description", 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.848 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.849 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.850 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.851 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.852 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.853 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "organization": { 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "organization", 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.854 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "proxy": { 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "proxy", 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.855 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "url": { 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "url", 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.856 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "test_orchestrator", 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.857 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.859 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.860 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.861 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "volumes", 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.862 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.863 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.864 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.865 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.866 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "max_concurrent_clones": { 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "max_concurrent_clones", 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "4", 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.867 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "periodic_async_work": { 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "periodic_async_work", 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.868 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Periodically check for async work", 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "snapshot_clone_delay": { 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "0", 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.869 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "True", 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.870 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix", 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "can_run": true, 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "error_string": "", 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "module_options": { 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "discovery_interval": { 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "discovery_interval", 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "uint", 2024-04-24T23:27:03.871 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "100", 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "identifier": { 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "identifier", 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.872 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "interval": { 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "interval", 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "secs", 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.873 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "60", 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "log_level": { 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_level", 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.874 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster": { 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster", 2024-04-24T23:27:03.875 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_cluster_level": { 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_cluster_level", 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.876 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "info", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [ 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "critical", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "debug", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "error", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "info", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "warning" 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.877 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "log_to_file": { 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "log_to_file", 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "bool", 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 1, 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "False", 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.878 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "zabbix_host": { 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix_host", 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "", 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.879 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "zabbix_port": { 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix_port", 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "int", 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "10051", 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.880 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "zabbix_sender": { 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "zabbix_sender", 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "str", 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "level": "advanced", 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "flags": 0, 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "min": "", 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "max": "", 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "enum_allowed": [], 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "desc": "", 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "long_desc": "", 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "tags": [], 2024-04-24T23:27:03.881 INFO:teuthology.orchestra.run.smithi102.stdout: "see_also": [] 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "services": { 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "dashboard": "https://172.21.15.102:8443/" 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "always_on_modules": { 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "octopus": [ 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "balancer", 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "devicehealth", 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "orchestrator", 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_autoscaler", 2024-04-24T23:27:03.882 INFO:teuthology.orchestra.run.smithi102.stdout: "progress", 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_support", 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: "status", 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: "telemetry", 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: "volumes" 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: "pacific": [ 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: "balancer", 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: "devicehealth", 2024-04-24T23:27:03.883 INFO:teuthology.orchestra.run.smithi102.stdout: "orchestrator", 2024-04-24T23:27:03.885 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_autoscaler", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "progress", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_support", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "status", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "telemetry", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "volumes" 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "quincy": [ 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "balancer", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "devicehealth", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "orchestrator", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_autoscaler", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "progress", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_support", 2024-04-24T23:27:03.886 INFO:teuthology.orchestra.run.smithi102.stdout: "status", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "telemetry", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "volumes" 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "reef": [ 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "balancer", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "devicehealth", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "orchestrator", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_autoscaler", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "progress", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_support", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "status", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "telemetry", 2024-04-24T23:27:03.887 INFO:teuthology.orchestra.run.smithi102.stdout: "volumes" 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "squid": [ 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "balancer", 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "devicehealth", 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "orchestrator", 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "pg_autoscaler", 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "progress", 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "rbd_support", 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "status", 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "telemetry", 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "volumes" 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "last_failure_osd_epoch": 91, 2024-04-24T23:27:03.888 INFO:teuthology.orchestra.run.smithi102.stdout: "active_clients": [ 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "devicehealth", 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v2", 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:0", 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 1672369231 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "libcephsqlite", 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-24T23:27:03.889 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v2", 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:0", 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 490505095 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "rbd_support", 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v2", 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:0", 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 175907817 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.890 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snap_schedule", 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v2", 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:0", 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 2769134552 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "snap_schedule", 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-24T23:27:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v2", 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:0", 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 4097065580 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: "name": "volumes", 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: "addrvec": [ 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v2", 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "172.21.15.102:0", 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 1997954243 2024-04-24T23:27:03.892 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.893 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.893 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:03.893 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-24T23:27:03.893 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:27:03.894 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:03 smithi161 ceph-mon[32709]: pgmap v437: 97 pgs: 97 active+clean; 19 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 5.3 MiB/s wr, 1.32k op/s 2024-04-24T23:27:03.894 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:03 smithi161 ceph-mon[32709]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:03.894 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:03 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3142842555' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:27:03.894 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:03 smithi183 ceph-mon[33137]: pgmap v437: 97 pgs: 97 active+clean; 19 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 5.3 MiB/s wr, 1.32k op/s 2024-04-24T23:27:03.894 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:03 smithi183 ceph-mon[33137]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:03.894 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:03 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3142842555' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:27:03.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.653+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fbfa40a2d00 msgr2=0x7fbfa40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:03.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.653+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fbfa40a2d00 0x7fbfa40a51c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8103a60 tx=0x7fbfa0005f90 comp rx=0 tx=0).stop 2024-04-24T23:27:03.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8101700 msgr2=0x7fbfb8197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:27:03.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8101700 0x7fbfb8197f70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005940 tx=0x7fbfa80040c0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 shutdown_connections 2024-04-24T23:27:03.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb8107710 0x7fbfb819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfb81020c0 0x7fbfb81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fbfa40a2d00 0x7fbfa40a51c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 --2- 172.21.15.102:0/3142842555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbfb8101700 0x7fbfb8197f70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:27:03.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 >> 172.21.15.102:0/3142842555 conn(0x7fbfb8078b40 msgr2=0x7fbfb8104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:27:03.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 shutdown_connections 2024-04-24T23:27:03.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:27:02.654+0000 7fbfbcb42640 1 -- 172.21.15.102:0/3142842555 wait complete. 2024-04-24T23:27:03.902 DEBUG:tasks.check_counter:Getting stats from a 2024-04-24T23:27:03.902 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-24T23:27:03.902 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:03 smithi102 ceph-mon[29216]: pgmap v437: 97 pgs: 97 active+clean; 19 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 5.3 MiB/s wr, 1.32k op/s 2024-04-24T23:27:03.902 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:03 smithi102 ceph-mon[29216]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:27:03.902 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:03 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3142842555' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:27:04.282 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:27:05.467 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:05 smithi102 ceph-mon[29216]: pgmap v438: 97 pgs: 97 active+clean; 19 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 3.2 MiB/s wr, 864 op/s 2024-04-24T23:27:05.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:05 smithi183 ceph-mon[33137]: pgmap v438: 97 pgs: 97 active+clean; 19 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 3.2 MiB/s wr, 864 op/s 2024-04-24T23:27:05.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:05 smithi161 ceph-mon[32709]: pgmap v438: 97 pgs: 97 active+clean; 19 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 3.2 MiB/s wr, 864 op/s 2024-04-24T23:27:06.749 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:27:06.749 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:27:06.749 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 275, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 262, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1598000, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 160848, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.045917745, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.014873081, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.034179992, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005048577, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 262, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.012204025, 2024-04-24T23:27:06.750 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000046580 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1598000, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 160848 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 2, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 199, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 320, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 190432, 2024-04-24T23:27:06.751 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 5, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.012712367, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.008441341, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.002069280, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 196, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.013608512, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000069431 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:06.752 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 190432 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 0, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 0, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.000253009, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000158007, 2024-04-24T23:27:06.753 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.754 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:27:06.755 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:27:06.756 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148977, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 30, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-24T23:27:06.757 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-24T23:27:06.758 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 273, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1556741, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 273, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-24T23:27:06.759 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 273, 2024-04-24T23:27:06.760 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1556741, 2024-04-24T23:27:06.760 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-24T23:27:06.760 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:06.760 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:06.760 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:06.760 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:06.760 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:06.760 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:27:07.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:07 smithi102 ceph-mon[29216]: pgmap v439: 97 pgs: 97 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 492 KiB/s wr, 233 op/s 2024-04-24T23:27:07.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:07 smithi102 ceph-mon[29216]: osdmap e106: 12 total, 12 up, 12 in 2024-04-24T23:27:07.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:07 smithi183 ceph-mon[33137]: pgmap v439: 97 pgs: 97 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 492 KiB/s wr, 233 op/s 2024-04-24T23:27:07.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:07 smithi183 ceph-mon[33137]: osdmap e106: 12 total, 12 up, 12 in 2024-04-24T23:27:07.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:07 smithi161 ceph-mon[32709]: pgmap v439: 97 pgs: 97 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 492 KiB/s wr, 233 op/s 2024-04-24T23:27:07.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:07 smithi161 ceph-mon[32709]: osdmap e106: 12 total, 12 up, 12 in 2024-04-24T23:27:09.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:09 smithi102 ceph-mon[29216]: pgmap v441: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 394 KiB/s wr, 267 op/s 2024-04-24T23:27:09.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:09 smithi183 ceph-mon[33137]: pgmap v441: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 394 KiB/s wr, 267 op/s 2024-04-24T23:27:09.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:09 smithi161 ceph-mon[32709]: pgmap v441: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 394 KiB/s wr, 267 op/s 2024-04-24T23:27:11.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:11 smithi183 ceph-mon[33137]: pgmap v442: 97 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 258 KiB/s wr, 276 op/s 2024-04-24T23:27:11.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:11 smithi102 ceph-mon[29216]: pgmap v442: 97 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 258 KiB/s wr, 276 op/s 2024-04-24T23:27:11.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:11 smithi161 ceph-mon[32709]: pgmap v442: 97 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 258 KiB/s wr, 276 op/s 2024-04-24T23:27:12.702 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-24T23:27:12.702 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-24T23:27:12.702 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-24T23:27:12.702 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-24T23:27:12.702 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-24T23:27:12.702 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-24T23:27:12.702 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-24T23:27:12.702 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-24T23:27:12.702 DEBUG:tasks.check_counter:Getting stats from d 2024-04-24T23:27:12.706 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-24T23:27:12.956 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:27:13.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:13 smithi183 ceph-mon[33137]: pgmap v443: 97 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 248 op/s 2024-04-24T23:27:13.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:13 smithi102 ceph-mon[29216]: pgmap v443: 97 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 248 op/s 2024-04-24T23:27:13.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:13 smithi161 ceph-mon[32709]: pgmap v443: 97 pgs: 12 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 63 active+clean; 18 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 248 op/s 2024-04-24T23:27:15.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:15 smithi183 ceph-mon[33137]: pgmap v444: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 KiB/s wr, 189 op/s 2024-04-24T23:27:15.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:15 smithi102 ceph-mon[29216]: pgmap v444: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 KiB/s wr, 189 op/s 2024-04-24T23:27:15.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:15 smithi161 ceph-mon[32709]: pgmap v444: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 KiB/s wr, 189 op/s 2024-04-24T23:27:17.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:17 smithi183 ceph-mon[33137]: pgmap v445: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 147 op/s 2024-04-24T23:27:17.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:17 smithi102 ceph-mon[29216]: pgmap v445: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 147 op/s 2024-04-24T23:27:17.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:17 smithi161 ceph-mon[32709]: pgmap v445: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 147 op/s 2024-04-24T23:27:19.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:19 smithi183 ceph-mon[33137]: pgmap v446: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.8 KiB/s wr, 127 op/s 2024-04-24T23:27:19.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:19 smithi102 ceph-mon[29216]: pgmap v446: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.8 KiB/s wr, 127 op/s 2024-04-24T23:27:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:19 smithi161 ceph-mon[32709]: pgmap v446: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.8 KiB/s wr, 127 op/s 2024-04-24T23:27:21.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:21 smithi161 ceph-mon[32709]: pgmap v447: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 KiB/s wr, 113 op/s 2024-04-24T23:27:21.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:21 smithi161 ceph-mon[32709]: osdmap e107: 12 total, 12 up, 12 in 2024-04-24T23:27:21.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:21 smithi183 ceph-mon[33137]: pgmap v447: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 KiB/s wr, 113 op/s 2024-04-24T23:27:21.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:21 smithi183 ceph-mon[33137]: osdmap e107: 12 total, 12 up, 12 in 2024-04-24T23:27:21.558 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:21 smithi102 ceph-mon[29216]: pgmap v447: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 KiB/s wr, 113 op/s 2024-04-24T23:27:21.558 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:21 smithi102 ceph-mon[29216]: osdmap e107: 12 total, 12 up, 12 in 2024-04-24T23:27:23.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:23 smithi161 ceph-mon[32709]: pgmap v449: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 614 B/s wr, 56 op/s 2024-04-24T23:27:23.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:23 smithi183 ceph-mon[33137]: pgmap v449: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 614 B/s wr, 56 op/s 2024-04-24T23:27:23.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:23 smithi102 ceph-mon[29216]: pgmap v449: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 614 B/s wr, 56 op/s 2024-04-24T23:27:25.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:25 smithi161 ceph-mon[32709]: pgmap v450: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 409 B/s wr, 19 op/s 2024-04-24T23:27:25.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:25 smithi183 ceph-mon[33137]: pgmap v450: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 409 B/s wr, 19 op/s 2024-04-24T23:27:25.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:25 smithi102 ceph-mon[29216]: pgmap v450: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 409 B/s wr, 19 op/s 2024-04-24T23:27:27.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:27 smithi161 ceph-mon[32709]: pgmap v451: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:27.375 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:27 smithi102 ceph-mon[29216]: pgmap v451: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:27.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:27 smithi183 ceph-mon[33137]: pgmap v451: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:29.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:29 smithi102 ceph-mon[29216]: pgmap v452: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:29.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:29 smithi161 ceph-mon[32709]: pgmap v452: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:29.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:29 smithi183 ceph-mon[33137]: pgmap v452: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:31.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:31 smithi161 ceph-mon[32709]: pgmap v453: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:31.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:31 smithi183 ceph-mon[33137]: pgmap v453: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:31.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:31 smithi102 ceph-mon[29216]: pgmap v453: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:33.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:33 smithi161 ceph-mon[32709]: pgmap v454: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:33.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:33 smithi183 ceph-mon[33137]: pgmap v454: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:33.567 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:33 smithi102 ceph-mon[29216]: pgmap v454: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 4, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 4, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 3040, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 448, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.001372574, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000030282, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000515572, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3, 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000644361, 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000214787 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:27:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 2, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 208, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 320, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 192016, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 5, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.012631557, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.008719999, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001822108, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 205, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.013342490, 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000065085 2024-04-24T23:27:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 192016 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 285, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 274, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1598928, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 168544, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-24T23:27:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.048328376, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.015346104, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.036286197, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005705530, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 273, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.013310665, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000048757 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1598928, 2024-04-24T23:27:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 168544 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:27:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:27:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 149017, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 31, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-24T23:27:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-24T23:27:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 287, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1558728, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 287, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 287, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1558728, 2024-04-24T23:27:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-24T23:27:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:33.578 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:27:35.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:35 smithi161 ceph-mon[32709]: pgmap v455: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-24T23:27:35.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:35 smithi183 ceph-mon[33137]: pgmap v455: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-24T23:27:35.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:35 smithi102 ceph-mon[29216]: pgmap v455: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-24T23:27:35.940 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-24T23:27:35.940 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-24T23:27:35.941 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-24T23:27:35.941 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-24T23:27:35.941 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-24T23:27:35.941 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-24T23:27:35.941 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-24T23:27:35.941 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-24T23:27:35.941 DEBUG:tasks.check_counter:Getting stats from g 2024-04-24T23:27:35.941 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-24T23:27:36.197 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:27:37.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:37 smithi102 ceph-mon[29216]: pgmap v456: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:27:37.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:37 smithi161 ceph-mon[32709]: pgmap v456: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:27:37.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:37 smithi183 ceph-mon[33137]: pgmap v456: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:27:37.906 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:27:37.906 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:27:37.906 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 0, 2024-04-24T23:27:37.906 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 0, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.001168587, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000280362, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:37.907 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 2, 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 206, 2024-04-24T23:27:37.908 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 320, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 191664, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 5, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.015561019, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.010759298, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001965109, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 203, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.008971015, 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000044192 2024-04-24T23:27:37.909 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.910 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:37.910 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:37.910 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:37.910 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:37.910 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.910 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T23:27:37.911 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 191664 2024-04-24T23:27:37.911 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.911 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:27:37.912 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 286, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 273, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1601152, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 167072, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.047235518, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.015709861, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.035391280, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005258529, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 273, 2024-04-24T23:27:37.913 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.016769651, 2024-04-24T23:27:37.914 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000061427 2024-04-24T23:27:37.914 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.914 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:37.914 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:37.914 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:37.914 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:37.914 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.914 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1601152, 2024-04-24T23:27:37.914 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 167072 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:27:37.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:27:37.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 149057, 2024-04-24T23:27:37.917 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 32, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-24T23:27:37.918 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 284, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1558338, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:37.919 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 284, 2024-04-24T23:27:37.920 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-24T23:27:37.920 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-24T23:27:37.920 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 284, 2024-04-24T23:27:37.920 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1558338, 2024-04-24T23:27:37.920 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-24T23:27:37.920 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:37.920 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:37.920 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:37.920 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:37.921 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:37.921 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:27:38.647 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-24T23:27:38.647 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-24T23:27:38.647 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-24T23:27:38.647 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-24T23:27:38.647 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-24T23:27:38.647 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-24T23:27:38.647 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-24T23:27:38.647 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-24T23:27:38.647 DEBUG:tasks.check_counter:Getting stats from j 2024-04-24T23:27:38.648 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-24T23:27:38.895 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:27:39.263 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:39 smithi102 ceph-mon[29216]: pgmap v457: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:27:39.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:39 smithi161 ceph-mon[32709]: pgmap v457: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:27:39.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:39 smithi183 ceph-mon[33137]: pgmap v457: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:27:40.322 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:40 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:40.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:40 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:40.600 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-24T23:27:40.600 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 287, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 276, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1599472, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 169824, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.049342549, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.014768285, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.037806514, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005496847, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 275, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.008749052, 2024-04-24T23:27:40.601 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000031814 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1599472, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 169824 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 6, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 213, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 3360, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 192992, 2024-04-24T23:27:40.602 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 5, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.013939951, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.009474851, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001852873, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 211, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.019636322, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000093063 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:40.603 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 192992 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 0, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 0, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.000343605, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000273278, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:40.604 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-24T23:27:40.605 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:27:40.606 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-24T23:27:40.607 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148967, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 29, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-24T23:27:40.608 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-24T23:27:40.609 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 289, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1558988, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 289, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 289, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1558988, 2024-04-24T23:27:40.610 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-24T23:27:40.611 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-24T23:27:40.611 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-24T23:27:40.611 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-24T23:27:40.611 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:40.611 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-24T23:27:40.611 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-24T23:27:40.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:40 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:41.270 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:41 smithi102 ceph-mon[29216]: pgmap v458: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:27:41.271 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:41 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:41.271 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:41 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:41.271 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:41 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:27:41.298 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-24T23:27:41.298 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-24T23:27:41.298 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-24T23:27:41.299 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-24T23:27:41.299 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-24T23:27:41.299 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-24T23:27:41.299 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-24T23:27:41.299 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-24T23:27:41.299 DEBUG:tasks.check_counter:Getting stats from b 2024-04-24T23:27:41.299 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-24T23:27:41.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:41 smithi183 ceph-mon[33137]: pgmap v458: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:27:41.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:41 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:41 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:41.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:41 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:27:41.598 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:27:41.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:41 smithi161 ceph-mon[32709]: pgmap v458: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:27:41.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:41 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:41.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:41 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:41.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:41 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' 2024-04-24T23:27:43.468 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:43 smithi161 ceph-mon[32709]: pgmap v459: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:27:43.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:43 smithi183 ceph-mon[33137]: pgmap v459: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:27:43.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:43 smithi102 ceph-mon[29216]: pgmap v459: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:27:43.943 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 6434, 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 4303, 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3005527, 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 8571770, 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 20, 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.479483698, 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.142564643, 2024-04-24T23:27:43.944 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.250990388, 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.113765125, 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4301, 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.491048234, 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000114170 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8409, 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001384997, 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000164 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 477568, 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 133200 2024-04-24T23:27:43.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 3807, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 3946, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1305448, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 3970781, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 133, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 8, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.352648373, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.098131746, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.184126311, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.077431288, 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:43.946 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3943, 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.724596095, 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000183767 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6656, 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000990220, 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000148 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 93264, 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 567712 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:27:43.947 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1857, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1904, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2322026, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1097353, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.214936447, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.078394036, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.097269176, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.045786571, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1904, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.170023700, 2024-04-24T23:27:43.948 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000089298 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2192, 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000416435, 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000189 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1726272, 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 182064 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-04-24T23:27:43.949 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1744, 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.984450377, 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000564478 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 165, 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.027183126, 2024-04-24T23:27:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000164746 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 1259, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 1134, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1134, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 8.229733354, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007257260 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 125, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-04-24T23:27:43.951 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 310, 2024-04-24T23:27:43.952 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-04-24T23:27:43.952 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-04-24T23:27:43.952 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 22, 2024-04-24T23:27:43.952 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 1, 2024-04-24T23:27:43.952 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 10, 2024-04-24T23:27:43.952 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 11, 2024-04-24T23:27:43.952 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 19, 2024-04-24T23:27:43.952 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 441, 2024-04-24T23:27:43.952 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 2, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 3, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 2079, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 1768, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 125, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 16, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 51, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 66, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 312, 2024-04-24T23:27:43.953 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 1, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 2, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:27:43.954 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 609, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 487, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 1064, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 192, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 69, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 683 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-04-24T23:27:43.955 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 128, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 120, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 68, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 71, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 266, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 1, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 153, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 91, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 61, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:27:43.956 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 66, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 1422, 2024-04-24T23:27:43.957 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 1398, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 1398, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 24, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 3, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 65, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 64, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 64, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 1, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 1, 2024-04-24T23:27:43.958 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 1, 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 10405354, 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 10532350, 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 509, 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 11.796528498, 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.023175890 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-04-24T23:27:43.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 22, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 374, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 352, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 21, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 157, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 136, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 22, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 629, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 607, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 3, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 407, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 404, 2024-04-24T23:27:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 60456, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 231804 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 1672, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 16, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 1259, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 102, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 53, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 236, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.389013794, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001648363 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-04-24T23:27:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 230, 2024-04-24T23:27:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.062095319, 2024-04-24T23:27:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000269979 2024-04-24T23:27:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-04-24T23:27:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 61, 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.146395682, 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002399929 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 220, 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.313766719, 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001426212 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-04-24T23:27:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-04-24T23:27:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 14, 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002655029, 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000189644 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 88, 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.105220595, 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001195688 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 14, 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.080621027, 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005758644 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8, 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004663975, 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000582996 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 17, 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.122781685, 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007222452 2024-04-24T23:27:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 80, 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 6.517406556, 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.081467581 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5, 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.125095020, 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.025019004 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8, 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.066866043, 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.008358255 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002575956, 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000858652 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8, 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.022596575, 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002824571 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002218265, 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000739421 2024-04-24T23:27:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 139, 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.265761114, 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001911950 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 6, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 4, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 108, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 54, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 850, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 360, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 360, 2024-04-24T23:27:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-04-24T23:27:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-04-24T23:27:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 386013, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 144, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 352, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 4, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-04-24T23:27:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8232, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 172, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 132324, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 707, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 1793, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 7508787, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 31, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 1793, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1793, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 30.094352511, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.016784357 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1793, 2024-04-24T23:27:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2480 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 1793, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 7, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 1786, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 290, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 27, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 7, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 516, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 221, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-04-24T23:27:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 605, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 54, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-04-24T23:27:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-04-24T23:27:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 107, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 22, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 737, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 23 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-04-24T23:27:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 10, 2024-04-24T23:27:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 91, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 91, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 12094, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 5459218, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 12094, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 12094, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 5459218, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-24T23:27:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 1793, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 7518026, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1793, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 7518026, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 1793, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 1793, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1793, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1793, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 91, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 9239, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 91, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 7, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 9239, 2024-04-24T23:27:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5597089d2500": { 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 1422, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 6338046, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 1422, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 509, 2024-04-24T23:27:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 6338046, 2024-04-24T23:27:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:43.990 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-24T23:27:44.823 INFO:tasks.check_counter:Daemon mds.b mds.exported=66 2024-04-24T23:27:44.824 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-04-24T23:27:44.824 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=128 2024-04-24T23:27:44.824 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=120 2024-04-24T23:27:44.824 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-24T23:27:44.824 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=14 2024-04-24T23:27:44.824 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=8 2024-04-24T23:27:44.824 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-04-24T23:27:44.824 DEBUG:tasks.check_counter:Getting stats from e 2024-04-24T23:27:44.824 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-24T23:27:45.071 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:27:45.425 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:45 smithi161 ceph-mon[32709]: pgmap v460: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-24T23:27:45.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:45 smithi183 ceph-mon[33137]: pgmap v460: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-24T23:27:45.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:45 smithi102 ceph-mon[29216]: pgmap v460: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-24T23:27:47.221 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-24T23:27:47.221 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:27:47.221 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 2936, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2938, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 901602, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 933655, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.222665666, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.067146821, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.055219795, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.083936105, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2936, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.103254834, 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000035168 2024-04-24T23:27:47.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 329, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 307, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1668796, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 176138, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 3, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-04-24T23:27:47.223 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.055123149, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.017670356, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.039784422, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.006615825, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 307, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.015754792, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000051318 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1668240, 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 175440 2024-04-24T23:27:47.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 3, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 211, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 588, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 532830, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.011717627, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.007633522, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.002065129, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000020928, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 208, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.020419098, 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000098168 2024-04-24T23:27:47.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 532512 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-24T23:27:47.226 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4898, 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.121317709, 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000024768 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-04-24T23:27:47.227 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 4, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 4, 2024-04-24T23:27:47.228 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 4, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-04-24T23:27:47.229 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:27:47.230 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-04-24T23:27:47.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-04-24T23:27:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-04-24T23:27:47.233 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 33, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-04-24T23:27:47.234 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4267239, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 4264634, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 33 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 4, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 24, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 20, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 2, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 17, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 15, 2024-04-24T23:27:47.235 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 4, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 29, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 25, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 53696, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215420 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-04-24T23:27:47.236 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.239 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.240 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.241 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.242 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.243 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.244 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.245 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:27:47.246 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 4, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 3, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 2610, 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-04-24T23:27:47.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:27:47.248 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:27:47.249 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 181331, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 48, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:27:47.250 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8232, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 172, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 16480, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 48, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-04-24T23:27:47.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 2939, 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 2939, 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2939, 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 7.284842830, 2024-04-24T23:27:47.252 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002478680 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2939, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2940 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 2939, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2939, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 1922, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1015, 2024-04-24T23:27:47.253 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-04-24T23:27:47.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-04-24T23:27:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 107, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 21, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 4, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 5, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 1, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-04-24T23:27:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-04-24T23:27:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-24T23:27:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 3264, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 2257715, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 3264, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3264, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2257715, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-04-24T23:27:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2939, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 72940, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2939, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 72940, 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-24T23:27:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2939, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2939, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2939, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2939, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:27:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x564a27b0e6e0": { 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-04-24T23:27:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:47.265 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-24T23:27:47.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:47 smithi161 ceph-mon[32709]: pgmap v461: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:27:47.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:47 smithi183 ceph-mon[33137]: pgmap v461: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:27:47.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:47 smithi102 ceph-mon[29216]: pgmap v461: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:27:48.061 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-24T23:27:48.062 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-24T23:27:48.062 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-24T23:27:48.062 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-24T23:27:48.062 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-24T23:27:48.062 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-24T23:27:48.062 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T23:27:48.062 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-24T23:27:48.062 DEBUG:tasks.check_counter:Getting stats from h 2024-04-24T23:27:48.062 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-24T23:27:48.334 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:27:49.375 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:49 smithi102 ceph-mon[29216]: pgmap v462: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:27:49.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:49 smithi183 ceph-mon[33137]: pgmap v462: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:27:49.526 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:49 smithi161 ceph-mon[32709]: pgmap v462: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:27:50.092 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-24T23:27:50.092 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:27:50.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 97, 2024-04-24T23:27:50.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 156, 2024-04-24T23:27:50.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 24319, 2024-04-24T23:27:50.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 168278, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 16, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.013669182, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.003600040, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.007146694, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.001315701, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 150, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.024502236, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000163348 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 133, 2024-04-24T23:27:50.093 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000031747, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000238 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 4096, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 5392 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1518, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1368, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2067013, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 676762, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 10, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.155947455, 2024-04-24T23:27:50.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.063726041, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.093888456, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.012321398, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1371, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.070897039, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000051711 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2126, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000405612, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000190 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1677584, 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 186080 2024-04-24T23:27:50.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 2878, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1585, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1657321, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 827237, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 11, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.137661198, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.053876670, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.118977806, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000991949, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1583, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.069771050, 2024-04-24T23:27:50.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000044075 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4140, 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000579948, 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000140 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 536192 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-04-24T23:27:50.097 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 102, 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.009472735, 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000092869 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 24, 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000791932, 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000032997 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.098 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 17, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 13, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 13, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.022631515, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001740885 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 4, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-04-24T23:27:50.099 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 14, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 1, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 13, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 13, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 187, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 3, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 3, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 288, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 194, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 1, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 83, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-04-24T23:27:50.100 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 3, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 9, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 3, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 6, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 3, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 117, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-04-24T23:27:50.101 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 3, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 5, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 5, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 2, 2024-04-24T23:27:50.102 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 7 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 3, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 425, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 254, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 256, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 6, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 3, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 3, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-04-24T23:27:50.103 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 3, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-04-24T23:27:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 33, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 33, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 2, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-04-24T23:27:50.105 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4267239, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 25, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.112296219, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004491848 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 19, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 117, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 98, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 17, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 198, 2024-04-24T23:27:50.106 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 181, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 14, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 204, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 190, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 3, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 13, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 10, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 57400, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223612 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 26, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 6, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 17, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 30, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 18, 2024-04-24T23:27:50.107 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5, 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001556879, 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000311375 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.108 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000325367, 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000162683 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-04-24T23:27:50.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-04-24T23:27:50.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.111 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.111 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.111 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-04-24T23:27:50.111 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.111 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.111 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.112 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-04-24T23:27:50.112 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.112 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.112 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.112 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-04-24T23:27:50.112 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.112 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.112 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001150168, 2024-04-24T23:27:50.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000383389 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.115 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.116 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.117 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.019599101, 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.006533033 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:50.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 4, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 2, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 1, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 858, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-04-24T23:27:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:27:50.120 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 328153, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 106, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8232, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 172, 2024-04-24T23:27:50.122 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 81296, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 306, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 86, 2024-04-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 85854, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 30, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 86, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 86, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.499989250, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005813828 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 86, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 116 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 86, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2, 2024-04-24T23:27:50.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 84, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 2, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 27, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 13, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 15, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-04-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-04-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 107, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-04-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 21, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 10, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 15, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 5, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 25, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 4 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-04-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 2, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 2, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 3, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 3, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 4489, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 3322720, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 4489, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4489, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 3322720, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-04-24T23:27:50.130 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 86, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 86301, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 86, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 86301, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 86, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 86, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 86, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 86, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.132 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 3, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 447, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 3, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2, 2024-04-24T23:27:50.133 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 447, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55679d704500": { 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 33, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 72935, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 33, 2024-04-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 25, 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 72935, 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-24T23:27:51.017 INFO:tasks.check_counter:Daemon mds.h mds.exported=3 2024-04-24T23:27:51.017 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-04-24T23:27:51.018 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=3 2024-04-24T23:27:51.018 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=425 2024-04-24T23:27:51.018 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-24T23:27:51.018 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-24T23:27:51.018 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T23:27:51.018 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-24T23:27:51.018 DEBUG:tasks.check_counter:Getting stats from k 2024-04-24T23:27:51.018 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-24T23:27:51.260 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:27:51.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:51 smithi161 ceph-mon[32709]: pgmap v463: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:27:51.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:51 smithi183 ceph-mon[33137]: pgmap v463: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:27:51.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:51 smithi102 ceph-mon[29216]: pgmap v463: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.001097714, 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:27:53.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000282041, 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:27:53.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 291, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 280, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1600560, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 172384, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.054456740, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.016921877, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.039458918, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.006748820, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 279, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.014640784, 2024-04-24T23:27:53.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000052475 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1600560, 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 172384 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 6, 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 216, 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3360, 2024-04-24T23:27:53.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 193520, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.014129010, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.009603402, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001972914, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 214, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.019074923, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000089135 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:53.097 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 193520 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-04-24T23:27:53.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-04-24T23:27:53.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-04-24T23:27:53.100 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148977, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 30, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-04-24T23:27:53.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-04-24T23:27:53.102 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 293, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1559508, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 293, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 293, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1559508, 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-24T23:27:53.103 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-24T23:27:53.104 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-24T23:27:53.104 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-24T23:27:53.104 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:53.104 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-24T23:27:53.104 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-24T23:27:53.358 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:53 smithi161 ceph-mon[32709]: pgmap v464: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:27:53.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:53 smithi183 ceph-mon[33137]: pgmap v464: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:27:53.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:53 smithi102 ceph-mon[29216]: pgmap v464: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:27:53.897 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-24T23:27:53.897 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-24T23:27:53.897 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-24T23:27:53.897 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-24T23:27:53.898 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-24T23:27:53.898 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-24T23:27:53.898 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-24T23:27:53.898 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-24T23:27:53.898 DEBUG:tasks.check_counter:Getting stats from c 2024-04-24T23:27:53.898 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-24T23:27:54.241 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:27:55.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:55 smithi183 ceph-mon[33137]: pgmap v465: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-24T23:27:55.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:55 smithi102 ceph-mon[29216]: pgmap v465: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-24T23:27:55.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:55 smithi161 ceph-mon[32709]: pgmap v465: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-24T23:27:56.755 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-24T23:27:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:27:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1881, 2024-04-24T23:27:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1860, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 6341552, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 669964, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.207766731, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.065048107, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.079501539, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.058019567, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1859, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.083580598, 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000044959 2024-04-24T23:27:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1665360, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 175152 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 543, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 750, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 136134, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 707200, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 8, 2024-04-24T23:27:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.061446813, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.023939880, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.013348501, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.017975729, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 748, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.039552577, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000052877 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 536368 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1089, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1091, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2420382, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 346309, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.095223811, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.033925814, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.022252293, 2024-04-24T23:27:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.031085984, 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1089, 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.045623934, 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000041895 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5290, 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.150127117, 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000028379 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-24T23:27:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 172, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 111, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 61, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 14, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-24T23:27:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-24T23:27:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:27:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-24T23:27:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:27:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 1422, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-24T23:27:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 64, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 10405354, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 10532350, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 10532289, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 1422 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 17, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 336, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 319, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 12, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 95, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 83, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 172, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 502, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 330, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-04-24T23:27:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 54704, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215420 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-24T23:27:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-24T23:27:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-24T23:27:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-24T23:27:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 6, 2024-04-24T23:27:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 4, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 5, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 3805, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:27:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:27:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:27:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 189433, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 52, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8232, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 172, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:27:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 207784, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 1085, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3182, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 26, 2024-04-24T23:27:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 4, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3182, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3182, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 6.521822272, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002049598 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3182, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3183 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3182, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3182, 2024-04-24T23:27:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 1991, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1188, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-24T23:27:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 2, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-24T23:27:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-24T23:27:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 107, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 21, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 6, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-24T23:27:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:27:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 3509, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 8562701, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 3509, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3509, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 8562701, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3182, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 6338056, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3182, 2024-04-24T23:27:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 6338056, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3182, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3182, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3182, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3182, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:27:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55f85bd546e0": { 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-24T23:27:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:27:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:27:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:27:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:27:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:27:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:27:56.800 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-24T23:27:57.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:57 smithi183 ceph-mon[33137]: pgmap v466: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:27:57.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:57 smithi102 ceph-mon[29216]: pgmap v466: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:27:57.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:57 smithi161 ceph-mon[32709]: pgmap v466: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T23:27:57.752 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-24T23:27:57.753 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-24T23:27:57.753 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-24T23:27:57.753 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-24T23:27:57.753 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-24T23:27:57.753 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-24T23:27:57.753 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T23:27:57.753 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-24T23:27:57.753 DEBUG:tasks.check_counter:Getting stats from f 2024-04-24T23:27:57.753 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-24T23:27:58.005 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:27:59.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:27:59 smithi183 ceph-mon[33137]: pgmap v467: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:27:59.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:27:59 smithi102 ceph-mon[29216]: pgmap v467: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:27:59.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:27:59 smithi161 ceph-mon[32709]: pgmap v467: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T23:28:00.057 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-24T23:28:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 478, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 458, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 5899138, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 224932, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.069932263, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.020611122, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.044903205, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.011345429, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 457, 2024-04-24T23:28:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.023070966, 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000050483 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1665744, 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 176272 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1313, 2024-04-24T23:28:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1522, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 21303544, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 953967, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 8, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.135435233, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.042315486, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.043889390, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.037411667, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1520, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.076843026, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000050554 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 538336 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2214, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2216, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 20298986, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 704110, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.218113578, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.060990327, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.069058484, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.070319449, 2024-04-24T23:28:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2214, 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.077946620, 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000035206 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-24T23:28:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6342, 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.155553050, 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000024527 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-24T23:28:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 2295, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 2295, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 2295, 2024-04-24T23:28:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 1, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-24T23:28:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-24T23:28:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-24T23:28:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-24T23:28:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-24T23:28:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 7631, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 14, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-04-24T23:28:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 49014843, 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 49012282, 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 7631 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-24T23:28:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 982, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 1684, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 702, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 122, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 146, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 24, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 2295, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 2333, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 38, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 58472, 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215420 2024-04-24T23:28:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-24T23:28:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-24T23:28:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-24T23:28:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.078 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.078 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-24T23:28:00.078 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.078 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.078 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.079 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-24T23:28:00.079 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.079 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.079 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.079 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.080 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-24T23:28:00.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.083 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-24T23:28:00.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.083 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.083 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-24T23:28:00.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.083 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-24T23:28:00.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.085 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.086 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 4, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 3, 2024-04-24T23:28:00.087 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 2642, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:28:00.088 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:28:00.089 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 215281, 2024-04-24T23:28:00.090 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 69, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8232, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 172, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-24T23:28:00.091 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 3938022, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 33250, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3674, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-04-24T23:28:00.092 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3674, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3674, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 11.715680261, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003188807 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3674, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3675 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3674, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3674, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-24T23:28:00.093 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 1934, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1738, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-24T23:28:00.094 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-24T23:28:00.095 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 107, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 21, 2024-04-24T23:28:00.096 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 8, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 9, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-04-24T23:28:00.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:00.098 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4001, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 47121963, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4001, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4001, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 47121963, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-24T23:28:00.099 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3674, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 44820544, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3674, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 44820544, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3674, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3674, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3674, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3674, 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:28:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-24T23:28:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-24T23:28:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x564658c0c6e0": { 2024-04-24T23:28:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:00.105 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-24T23:28:00.889 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-24T23:28:00.889 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-24T23:28:00.889 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-24T23:28:00.890 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-24T23:28:00.890 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-24T23:28:00.890 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-24T23:28:00.890 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T23:28:00.890 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-24T23:28:00.890 DEBUG:tasks.check_counter:Getting stats from i 2024-04-24T23:28:00.890 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-24T23:28:01.131 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:28:01.398 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:01 smithi183 ceph-mon[33137]: pgmap v468: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:28:01.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:01 smithi102 ceph-mon[29216]: pgmap v468: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:28:01.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:01 smithi161 ceph-mon[32709]: pgmap v468: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T23:28:02.909 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 15191, 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 20945, 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 6273655, 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 22829221, 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 14, 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 1.020442201, 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.391974030, 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.689452250, 2024-04-24T23:28:02.910 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.047077202, 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 20945, 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.795664274, 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000037988 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 31184, 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004902548, 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000157 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1676032, 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 181328 2024-04-24T23:28:02.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3439, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 3655, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 896766, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 22995402, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 13, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.503634110, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.118733805, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.077661694, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.282315818, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3648, 2024-04-24T23:28:02.912 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.189092446, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000051834 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000001931, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000120 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2656, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 629776 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 5535, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 6848, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1809043, 2024-04-24T23:28:02.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 14228054, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 8, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.406924403, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.146715912, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.172951646, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.082075129, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6850, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.358718593, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000052367 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:28:02.914 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6997, 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001227004, 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000175 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3616, 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 16992 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-24T23:28:02.915 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5811, 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.016498863, 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000519101 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4366, 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.184108569, 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000042168 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-24T23:28:02.916 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 6960, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 6949, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6949, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 52.643614553, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.007575710 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 11, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 52, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 5, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-24T23:28:02.917 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 1610, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 237, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 1280, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 93, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 1270, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 10, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 1, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 1, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 4, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 11751, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 5546, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 1, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 73, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:28:02.918 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 1, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 1, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 2, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 66, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 315, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 3, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 117, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-04-24T23:28:02.919 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:28:02.920 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-24T23:28:02.920 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:28:02.920 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:28:02.920 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:28:02.920 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:28:02.921 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:28:02.921 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-24T23:28:02.921 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 116, 2024-04-24T23:28:02.921 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 871, 2024-04-24T23:28:02.921 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 3503, 2024-04-24T23:28:02.921 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 6, 2024-04-24T23:28:02.921 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:28:02.921 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 5223, 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 3458, 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 17, 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 2191 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 156, 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 192, 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 128, 2024-04-24T23:28:02.922 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 141, 2024-04-24T23:28:02.923 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 196, 2024-04-24T23:28:02.923 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 83, 2024-04-24T23:28:02.923 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-24T23:28:02.923 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:28:02.923 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 778, 2024-04-24T23:28:02.923 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 695, 2024-04-24T23:28:02.923 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-24T23:28:02.923 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 1, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 5, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-24T23:28:02.924 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-24T23:28:02.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.925 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-24T23:28:02.925 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-24T23:28:02.925 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 7631, 2024-04-24T23:28:02.925 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-24T23:28:02.925 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 7631, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 14, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 14, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 49014843, 2024-04-24T23:28:02.926 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2632, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 34.144639167, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.012972887 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 996, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 1695, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 699, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 137, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 161, 2024-04-24T23:28:02.927 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 24, 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 1610, 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 2555, 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 945, 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 1, 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 782, 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 781, 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 69396, 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223612 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 7988, 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 24, 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 6960, 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 31, 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 79, 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 529, 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.549817790, 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001039353 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1030, 2024-04-24T23:28:02.929 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.312490925, 2024-04-24T23:28:02.930 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000303389 2024-04-24T23:28:02.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.930 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-24T23:28:02.930 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.930 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.930 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.930 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-24T23:28:02.930 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.931 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.931 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.931 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-24T23:28:02.931 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3320, 2024-04-24T23:28:02.931 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.722753592, 2024-04-24T23:28:02.931 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000217696 2024-04-24T23:28:02.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.931 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-24T23:28:02.933 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.934 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.934 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.934 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-24T23:28:02.934 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 17, 2024-04-24T23:28:02.934 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.013565189, 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000797952 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-24T23:28:02.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1035, 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 47.597206429, 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.045987639 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 216, 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.648182123, 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003000843 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 33, 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.062128518, 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001882682 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.936 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 94, 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.621389791, 2024-04-24T23:28:02.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.017248827 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-24T23:28:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002831929, 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002831929 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4, 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001807469, 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000451867 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10, 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.007223059, 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000722305 2024-04-24T23:28:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 660, 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.104217739, 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001673057 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 4, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 1304, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 652, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 871, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:28:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:28:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 956924, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 281, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-24T23:28:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8232, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 172, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 3557163, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 37303, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-24T23:28:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 7230, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 45400459, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 28, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 7230, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7230, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 183.703053656, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.025408444 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-24T23:28:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7230, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 8124 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 7230, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 28, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 7202, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 28, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 2660, 2024-04-24T23:28:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 74, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 1, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 3819, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:28:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 6, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-24T23:28:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 107, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 21, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-04-24T23:28:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 1093, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 413 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 2 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-24T23:28:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 51, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 10, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 3793, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 695, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 24161, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 6782939, 2024-04-24T23:28:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 24161, 2024-04-24T23:28:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:28:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:28:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 24161, 2024-04-24T23:28:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 6782939, 2024-04-24T23:28:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 7230, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 45485766, 2024-04-24T23:28:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 7230, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 45485766, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 7230, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 7230, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 7230, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 7230, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 695, 2024-04-24T23:28:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 85307, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 695, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 28, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 85307, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55f03cc5c500": { 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 7631, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 44820539, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 7631, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2632, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 44820539, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:02.960 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-24T23:28:03.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:03 smithi183 ceph-mon[33137]: pgmap v469: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:28:03.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:03 smithi102 ceph-mon[29216]: pgmap v469: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:28:03.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:03 smithi161 ceph-mon[32709]: pgmap v469: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T23:28:03.727 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-04-24T23:28:03.727 INFO:tasks.check_counter:Daemon mds.i mds.imported=66 2024-04-24T23:28:03.727 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=156 2024-04-24T23:28:03.727 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=192 2024-04-24T23:28:03.727 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-24T23:28:03.727 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-24T23:28:03.727 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T23:28:03.728 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=5 2024-04-24T23:28:03.728 DEBUG:tasks.check_counter:Getting stats from l 2024-04-24T23:28:03.728 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 97eecd66-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-24T23:28:03.968 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:28:05.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:05 smithi161 ceph-mon[32709]: pgmap v470: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-24T23:28:05.415 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:05 smithi183 ceph-mon[33137]: pgmap v470: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-24T23:28:05.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:05 smithi102 ceph-mon[29216]: pgmap v470: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-24T23:28:05.830 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-24T23:28:05.830 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:28:05.830 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 295, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 284, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1601648, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 174944, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.053240747, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.017042174, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.037250222, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.006761161, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 283, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.014497958, 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000051229 2024-04-24T23:28:05.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1601648, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 174944 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 7, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 220, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3520, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 272192, 2024-04-24T23:28:05.832 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.014583924, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.009854688, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002137332, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 218, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.015296092, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000070165 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.833 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 272192 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 0, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 0, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.000319398, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.000257286, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:28:05.834 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-24T23:28:05.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:28:05.836 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-24T23:28:05.837 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 147970, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 29, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:28:05.838 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 928, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:28:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 298, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1560055, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 298, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 298, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1560055, 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-24T23:28:05.840 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-24T23:28:05.841 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-24T23:28:05.841 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-24T23:28:05.841 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:05.841 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-24T23:28:05.841 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-24T23:28:06.568 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-24T23:28:06.568 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-24T23:28:06.568 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-24T23:28:06.568 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-24T23:28:06.568 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-24T23:28:06.569 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-24T23:28:06.569 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-24T23:28:06.569 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-24T23:28:06.569 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-24T23:28:06.580 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-24T23:28:06.580 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:28:06.581 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:06.613 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:28:06.614 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:06.673 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:28:06.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 -- 172.21.15.102:0/3513346260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 msgr2=0x7f0dc4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:06.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 --2- 172.21.15.102:0/3513346260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 0x7f0dc4100ac0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009930 tx=0x7f0db002a420 comp rx=0 tx=0).stop 2024-04-24T23:28:06.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 -- 172.21.15.102:0/3513346260 shutdown_connections 2024-04-24T23:28:06.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 --2- 172.21.15.102:0/3513346260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dc4101000 0x7f0dc410d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:06.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 --2- 172.21.15.102:0/3513346260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 0x7f0dc4100ac0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:06.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 --2- 172.21.15.102:0/3513346260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dc4106650 0x7f0dc4106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:06.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 -- 172.21.15.102:0/3513346260 >> 172.21.15.102:0/3513346260 conn(0x7f0dc40769c0 msgr2=0x7f0dc4076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:06.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 -- 172.21.15.102:0/3513346260 shutdown_connections 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 -- 172.21.15.102:0/3513346260 wait complete. 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 Processor -- start 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.825+0000 7f0dcb015640 1 -- start start 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dcb015640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 0x7f0dc4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dcb015640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dc4101000 0x7f0dc4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dcb015640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dc4106650 0x7f0dc419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dcb015640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4111a10 con 0x7f0dc4100640 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dcb015640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4111890 con 0x7f0dc4106650 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dcb015640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4111b90 con 0x7f0dc4101000 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc8db1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 0x7f0dc4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc8db1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 0x7f0dc4197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53772/0 (socket says 172.21.15.102:53772) 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc8db1640 1 -- 172.21.15.102:0/1032462302 learned_addr learned my addr 172.21.15.102:0/1032462302 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc95b2640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dc4106650 0x7f0dc419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:06.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc3fff640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dc4101000 0x7f0dc4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc8db1640 1 -- 172.21.15.102:0/1032462302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dc4101000 msgr2=0x7f0dc4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc8db1640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dc4101000 0x7f0dc4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc8db1640 1 -- 172.21.15.102:0/1032462302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dc4106650 msgr2=0x7f0dc419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc8db1640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dc4106650 0x7f0dc419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc8db1640 1 -- 172.21.15.102:0/1032462302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc419cfc0 con 0x7f0dc4100640 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc95b2640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dc4106650 0x7f0dc419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc8db1640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 0x7f0dc4197fd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b4a0 tx=0x7f0db800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc1ffb640 1 -- 172.21.15.102:0/1032462302 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8014070 con 0x7f0dc4100640 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc1ffb640 1 -- 172.21.15.102:0/1032462302 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db8010ac0 con 0x7f0dc4100640 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc419d2b0 con 0x7f0dc4100640 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.826+0000 7f0dc1ffb640 1 -- 172.21.15.102:0/1032462302 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8017c30 con 0x7f0dc4100640 2024-04-24T23:28:06.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.827+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc41a4b40 con 0x7f0dc4100640 2024-04-24T23:28:06.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.828+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc40689b0 con 0x7f0dc4100640 2024-04-24T23:28:06.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.828+0000 7f0dc1ffb640 1 -- 172.21.15.102:0/1032462302 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f0db8017070 con 0x7f0dc4100640 2024-04-24T23:28:06.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.829+0000 7f0dc1ffb640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0db40a2d80 0x7f0db40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:06.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.829+0000 7f0dc3fff640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0db40a2d80 0x7f0db40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:06.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.829+0000 7f0dc1ffb640 1 -- 172.21.15.102:0/1032462302 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7f0db80ce750 con 0x7f0dc4100640 2024-04-24T23:28:06.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.830+0000 7f0dc3fff640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0db40a2d80 0x7f0db40a5240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000c00 tx=0x7f0db0031400 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:06.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:06.834+0000 7f0dc1ffb640 1 -- 172.21.15.102:0/1032462302 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0db809b440 con 0x7f0dc4100640 2024-04-24T23:28:07.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.011+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dc4108bd0 con 0x7f0dc4100640 2024-04-24T23:28:07.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.012+0000 7f0dc1ffb640 1 -- 172.21.15.102:0/1032462302 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+23244 (secure 0 0 0) 0x7f0db809ad40 con 0x7f0dc4100640 2024-04-24T23:28:07.014 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:28:07.014 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:27:20.075691+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/2090298312":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/835175442":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/3611055373":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/4149172465":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/1812533171":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/3611055373":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:28:07.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.014+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0db40a2d80 msgr2=0x7f0db40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0db40a2d80 0x7f0db40a5240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000c00 tx=0x7f0db0031400 comp rx=0 tx=0).stop 2024-04-24T23:28:07.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 msgr2=0x7f0dc4197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 0x7f0dc4197fd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b4a0 tx=0x7f0db800b960 comp rx=0 tx=0).stop 2024-04-24T23:28:07.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 shutdown_connections 2024-04-24T23:28:07.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dc4106650 0x7f0dc419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dc4101000 0x7f0dc4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f0db40a2d80 0x7f0db40a5240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 --2- 172.21.15.102:0/1032462302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc4100640 0x7f0dc4197fd0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 >> 172.21.15.102:0/1032462302 conn(0x7f0dc40769c0 msgr2=0x7f0dc40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 shutdown_connections 2024-04-24T23:28:07.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.015+0000 7f0dcb015640 1 -- 172.21.15.102:0/1032462302 wait complete. 2024-04-24T23:28:07.027 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:58.730586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:59.855279+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:28:07.028 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:28:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.130+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/1179739393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f01077a0 msgr2=0x7fd6f0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.130+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/1179739393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f01077a0 0x7fd6f0107b80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009930 tx=0x7fd6e402a420 comp rx=0 tx=0).stop 2024-04-24T23:28:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.130+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/1179739393 shutdown_connections 2024-04-24T23:28:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.130+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/1179739393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f0102150 0x7fd6f010e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.130+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/1179739393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6f0101790 0x7fd6f0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.130+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/1179739393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f01077a0 0x7fd6f0107b80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.130+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/1179739393 >> 172.21.15.102:0/1179739393 conn(0x7fd6f00fd2a0 msgr2=0x7fd6f00ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.130+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/1179739393 shutdown_connections 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.130+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/1179739393 wait complete. 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6f5c5d640 1 Processor -- start 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6f5c5d640 1 -- start start 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6f5c5d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6f0101790 0x7fd6f0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6f5c5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0102150 0x7fd6f01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6f5c5d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f01077a0 0x7fd6f019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6f5c5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0111a60 con 0x7fd6f0102150 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6f5c5d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01118e0 con 0x7fd6f01077a0 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6f5c5d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0111be0 con 0x7fd6f0101790 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6eeffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0102150 0x7fd6f01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6eeffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0102150 0x7fd6f01985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53786/0 (socket says 172.21.15.102:53786) 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.131+0000 7fd6eeffd640 1 -- 172.21.15.102:0/3527729986 learned_addr learned my addr 172.21.15.102:0/3527729986 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:07.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6ef7fe640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6f0101790 0x7fd6f0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6effff640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f01077a0 0x7fd6f019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6eeffd640 1 -- 172.21.15.102:0/3527729986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6f0101790 msgr2=0x7fd6f0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6eeffd640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6f0101790 0x7fd6f0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6eeffd640 1 -- 172.21.15.102:0/3527729986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f01077a0 msgr2=0x7fd6f019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6eeffd640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f01077a0 0x7fd6f019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6eeffd640 1 -- 172.21.15.102:0/3527729986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f019d050 con 0x7fd6f0102150 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6eeffd640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0102150 0x7fd6f01985a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000b4a0 tx=0x7fd6e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6ecff9640 1 -- 172.21.15.102:0/3527729986 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0014070 con 0x7fd6f0102150 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6ecff9640 1 -- 172.21.15.102:0/3527729986 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6e0010ac0 con 0x7fd6f0102150 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f019d340 con 0x7fd6f0102150 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6ecff9640 1 -- 172.21.15.102:0/3527729986 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0017c30 con 0x7fd6f0102150 2024-04-24T23:28:07.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.132+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a4bd0 con 0x7fd6f0102150 2024-04-24T23:28:07.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.133+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f0068560 con 0x7fd6f0102150 2024-04-24T23:28:07.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.135+0000 7fd6ecff9640 1 -- 172.21.15.102:0/3527729986 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd6e0017070 con 0x7fd6f0102150 2024-04-24T23:28:07.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.135+0000 7fd6ecff9640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6d80a2d00 0x7fd6d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.136+0000 7fd6ef7fe640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6d80a2d00 0x7fd6d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.136+0000 7fd6ecff9640 1 -- 172.21.15.102:0/3527729986 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7fd6e00ceab0 con 0x7fd6f0102150 2024-04-24T23:28:07.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.136+0000 7fd6ef7fe640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6d80a2d00 0x7fd6d80a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009900 tx=0x7fd6e4000bc0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.138+0000 7fd6ecff9640 1 -- 172.21.15.102:0/3527729986 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd6e009b150 con 0x7fd6f0102150 2024-04-24T23:28:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:07 smithi102 ceph-mon[29216]: pgmap v471: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-24T23:28:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:07 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1032462302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:07.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.323+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6f0109d00 con 0x7fd6f0102150 2024-04-24T23:28:07.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.323+0000 7fd6ecff9640 1 -- 172.21.15.102:0/3527729986 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7fd6e009f450 con 0x7fd6f0102150 2024-04-24T23:28:07.324 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6d80a2d00 msgr2=0x7fd6d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6d80a2d00 0x7fd6d80a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009900 tx=0x7fd6e4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0102150 msgr2=0x7fd6f01985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0102150 0x7fd6f01985a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000b4a0 tx=0x7fd6e000b960 comp rx=0 tx=0).stop 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 shutdown_connections 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6f01077a0 0x7fd6f019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd6d80a2d00 0x7fd6d80a51c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0102150 0x7fd6f01985a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 --2- 172.21.15.102:0/3527729986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6f0101790 0x7fd6f0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.325+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 >> 172.21.15.102:0/3527729986 conn(0x7fd6f00fd2a0 msgr2=0x7fd6f00fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.326+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 shutdown_connections 2024-04-24T23:28:07.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.326+0000 7fd6f5c5d640 1 -- 172.21.15.102:0/3527729986 wait complete. 2024-04-24T23:28:07.338 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:28:07.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.441+0000 7f4234b5a640 1 -- 172.21.15.102:0/1563929717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230106630 msgr2=0x7f4230106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.441+0000 7f4234b5a640 1 --2- 172.21.15.102:0/1563929717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230106630 0x7f4230106a10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4220009930 tx=0x7f422002a420 comp rx=0 tx=0).stop 2024-04-24T23:28:07.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.441+0000 7f4234b5a640 1 -- 172.21.15.102:0/1563929717 shutdown_connections 2024-04-24T23:28:07.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.441+0000 7f4234b5a640 1 --2- 172.21.15.102:0/1563929717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4230100fe0 0x7f423010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.441+0000 7f4234b5a640 1 --2- 172.21.15.102:0/1563929717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230100620 0x7f4230100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.441+0000 7f4234b5a640 1 --2- 172.21.15.102:0/1563929717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230106630 0x7f4230106a10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.441+0000 7f4234b5a640 1 -- 172.21.15.102:0/1563929717 >> 172.21.15.102:0/1563929717 conn(0x7f42300769c0 msgr2=0x7f4230076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.441+0000 7f4234b5a640 1 -- 172.21.15.102:0/1563929717 shutdown_connections 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.441+0000 7f4234b5a640 1 -- 172.21.15.102:0/1563929717 wait complete. 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f4234b5a640 1 Processor -- start 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f4234b5a640 1 -- start start 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f4234b5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4230100620 0x7f4230198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f4234b5a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230100fe0 0x7f4230199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f4234b5a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230106630 0x7f42301a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f4234b5a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f42301108b0 con 0x7f4230100fe0 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f4234b5a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4230110730 con 0x7f4230106630 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f4234b5a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4230110a30 con 0x7f4230100620 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f422dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230100fe0 0x7f4230199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f422dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230100fe0 0x7f4230199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53796/0 (socket says 172.21.15.102:53796) 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f422dd9b640 1 -- 172.21.15.102:0/4136366028 learned_addr learned my addr 172.21.15.102:0/4136366028 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:07.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f422ed9d640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230106630 0x7f42301a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f422e59c640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4230100620 0x7f4230198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f422dd9b640 1 -- 172.21.15.102:0/4136366028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4230100620 msgr2=0x7f4230198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.442+0000 7f422dd9b640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4230100620 0x7f4230198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f422dd9b640 1 -- 172.21.15.102:0/4136366028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230106630 msgr2=0x7f42301a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f422dd9b640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230106630 0x7f42301a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f422dd9b640 1 -- 172.21.15.102:0/4136366028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42301a43a0 con 0x7f4230100fe0 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f422ed9d640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230106630 0x7f42301a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f422dd9b640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230100fe0 0x7f4230199320 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f422400e9f0 tx=0x7f422400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f421f7fe640 1 -- 172.21.15.102:0/4136366028 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422400cd80 con 0x7f4230100fe0 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f421f7fe640 1 -- 172.21.15.102:0/4136366028 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42240108e0 con 0x7f4230100fe0 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42301a4570 con 0x7f4230100fe0 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f421f7fe640 1 -- 172.21.15.102:0/4136366028 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224018920 con 0x7f4230100fe0 2024-04-24T23:28:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.443+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42301a4a90 con 0x7f4230100fe0 2024-04-24T23:28:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.444+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4230101d60 con 0x7f4230100fe0 2024-04-24T23:28:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.446+0000 7f421f7fe640 1 -- 172.21.15.102:0/4136366028 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f4224010a90 con 0x7f4230100fe0 2024-04-24T23:28:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.446+0000 7f421f7fe640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f42180a2d00 0x7f42180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.446+0000 7f421f7fe640 1 -- 172.21.15.102:0/4136366028 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7f42240cfdf0 con 0x7f4230100fe0 2024-04-24T23:28:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.447+0000 7f422e59c640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f42180a2d00 0x7f42180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.447+0000 7f422e59c640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f42180a2d00 0x7f42180a51c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4220000c00 tx=0x7f4220032ba0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.449+0000 7f421f7fe640 1 -- 172.21.15.102:0/4136366028 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f422409c9d0 con 0x7f4230100fe0 2024-04-24T23:28:07.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:07 smithi183 ceph-mon[33137]: pgmap v471: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-24T23:28:07.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:07 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1032462302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:07 smithi161 ceph-mon[32709]: pgmap v471: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-24T23:28:07.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:07 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1032462302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.635+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4230108b90 con 0x7f4230100fe0 2024-04-24T23:28:07.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.635+0000 7f421f7fe640 1 -- 172.21.15.102:0/4136366028 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f422409c2d0 con 0x7f4230100fe0 2024-04-24T23:28:07.636 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-24T23:28:07.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f42180a2d00 msgr2=0x7f42180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f42180a2d00 0x7f42180a51c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4220000c00 tx=0x7f4220032ba0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230100fe0 msgr2=0x7f4230199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230100fe0 0x7f4230199320 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f422400e9f0 tx=0x7f422400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 shutdown_connections 2024-04-24T23:28:07.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230106630 0x7f42301a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f42180a2d00 0x7f42180a51c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230100fe0 0x7f4230199320 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 --2- 172.21.15.102:0/4136366028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4230100620 0x7f4230198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 >> 172.21.15.102:0/4136366028 conn(0x7f42300769c0 msgr2=0x7f42300fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 shutdown_connections 2024-04-24T23:28:07.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.637+0000 7f4234b5a640 1 -- 172.21.15.102:0/4136366028 wait complete. 2024-04-24T23:28:07.650 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 -- 172.21.15.102:0/3276165129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100610 msgr2=0x7f9e0c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 --2- 172.21.15.102:0/3276165129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100610 0x7f9e0c100a90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009930 tx=0x7f9dfc02a420 comp rx=0 tx=0).stop 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 -- 172.21.15.102:0/3276165129 shutdown_connections 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 --2- 172.21.15.102:0/3276165129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e0c100fd0 0x7f9e0c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 --2- 172.21.15.102:0/3276165129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100610 0x7f9e0c100a90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 --2- 172.21.15.102:0/3276165129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e0c1066b0 0x7f9e0c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 -- 172.21.15.102:0/3276165129 >> 172.21.15.102:0/3276165129 conn(0x7f9e0c077740 msgr2=0x7f9e0c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 -- 172.21.15.102:0/3276165129 shutdown_connections 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 -- 172.21.15.102:0/3276165129 wait complete. 2024-04-24T23:28:07.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.753+0000 7f9e135ce640 1 Processor -- start 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e135ce640 1 -- start start 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e135ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e0c100610 0x7f9e0c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e135ce640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100fd0 0x7f9e0c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e135ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e0c1066b0 0x7f9e0c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e135ce640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1108e0 con 0x7f9e0c100fd0 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e135ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c110760 con 0x7f9e0c100610 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e135ce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c110a60 con 0x7f9e0c1066b0 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e10b69640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100fd0 0x7f9e0c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e10b69640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100fd0 0x7f9e0c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53808/0 (socket says 172.21.15.102:53808) 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e10b69640 1 -- 172.21.15.102:0/2907783796 learned_addr learned my addr 172.21.15.102:0/2907783796 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e1136a640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e0c100610 0x7f9e0c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e11b6b640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e0c1066b0 0x7f9e0c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e10b69640 1 -- 172.21.15.102:0/2907783796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e0c1066b0 msgr2=0x7f9e0c19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e10b69640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e0c1066b0 0x7f9e0c19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e10b69640 1 -- 172.21.15.102:0/2907783796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e0c100610 msgr2=0x7f9e0c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e10b69640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e0c100610 0x7f9e0c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e10b69640 1 -- 172.21.15.102:0/2907783796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c19cf40 con 0x7f9e0c100fd0 2024-04-24T23:28:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e10b69640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100fd0 0x7f9e0c198490 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc004770 tx=0x7f9dfc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e027fc640 1 -- 172.21.15.102:0/2907783796 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc03f070 con 0x7f9e0c100fd0 2024-04-24T23:28:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.754+0000 7f9e027fc640 1 -- 172.21.15.102:0/2907783796 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dfc03b460 con 0x7f9e0c100fd0 2024-04-24T23:28:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.755+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0c19d1d0 con 0x7f9e0c100fd0 2024-04-24T23:28:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.755+0000 7f9e027fc640 1 -- 172.21.15.102:0/2907783796 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc035070 con 0x7f9e0c100fd0 2024-04-24T23:28:07.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.755+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0c1a4ac0 con 0x7f9e0c100fd0 2024-04-24T23:28:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.756+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e0c068560 con 0x7f9e0c100fd0 2024-04-24T23:28:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.757+0000 7f9e027fc640 1 -- 172.21.15.102:0/2907783796 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f9dfc03b610 con 0x7f9e0c100fd0 2024-04-24T23:28:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.757+0000 7f9e027fc640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9df80a2d80 0x7f9df80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.758+0000 7f9e1136a640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9df80a2d80 0x7f9df80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.758+0000 7f9e027fc640 1 -- 172.21.15.102:0/2907783796 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7f9dfc0ef7e0 con 0x7f9e0c100fd0 2024-04-24T23:28:07.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.758+0000 7f9e1136a640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9df80a2d80 0x7f9df80a5240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c101750 tx=0x7f9e0800b390 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:07.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.762+0000 7f9e027fc640 1 -- 172.21.15.102:0/2907783796 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9dfc0bc360 con 0x7f9e0c100fd0 2024-04-24T23:28:07.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.945+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e0c1a4db0 con 0x7f9e0c100fd0 2024-04-24T23:28:07.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.945+0000 7f9e027fc640 1 -- 172.21.15.102:0/2907783796 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7f9dfc0bbc60 con 0x7f9e0c100fd0 2024-04-24T23:28:07.946 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-24T23:28:07.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9df80a2d80 msgr2=0x7f9df80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9df80a2d80 0x7f9df80a5240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c101750 tx=0x7f9e0800b390 comp rx=0 tx=0).stop 2024-04-24T23:28:07.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100fd0 msgr2=0x7f9e0c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:07.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100fd0 0x7f9e0c198490 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc004770 tx=0x7f9dfc0042f0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 shutdown_connections 2024-04-24T23:28:07.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e0c1066b0 0x7f9e0c19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f9df80a2d80 0x7f9df80a5240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e0c100fd0 0x7f9e0c198490 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 --2- 172.21.15.102:0/2907783796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e0c100610 0x7f9e0c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:07.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 >> 172.21.15.102:0/2907783796 conn(0x7f9e0c077740 msgr2=0x7f9e0c0ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:07.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 shutdown_connections 2024-04-24T23:28:07.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:07.947+0000 7f9e135ce640 1 -- 172.21.15.102:0/2907783796 wait complete. 2024-04-24T23:28:07.959 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:28:08.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1433202625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b41020e0 msgr2=0x7fc9b410e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1433202625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b41020e0 0x7fc9b410e5f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009cd0 tx=0x7fc9ac02a8e0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1433202625 shutdown_connections 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1433202625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b41020e0 0x7fc9b410e5f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1433202625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9b4101720 0x7fc9b4101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1433202625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b4107730 0x7fc9b4107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1433202625 >> 172.21.15.102:0/1433202625 conn(0x7fc9b4078b40 msgr2=0x7fc9b4078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1433202625 shutdown_connections 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1433202625 wait complete. 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 Processor -- start 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.060+0000 7fc9bc4ee640 1 -- start start 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9bc4ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b4101720 0x7fc9b4197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9bc4ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9b41020e0 0x7fc9b4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9bc4ee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b4107730 0x7fc9b419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9bc4ee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41119f0 con 0x7fc9b4107730 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9bc4ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4111870 con 0x7fc9b41020e0 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9bc4ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4111b70 con 0x7fc9b4101720 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9baa8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b4107730 0x7fc9b419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9baa8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b4107730 0x7fc9b419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53818/0 (socket says 172.21.15.102:53818) 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9baa8b640 1 -- 172.21.15.102:0/1027113649 learned_addr learned my addr 172.21.15.102:0/1027113649 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9b9a89640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9b41020e0 0x7fc9b4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9ba28a640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b4101720 0x7fc9b4197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9baa8b640 1 -- 172.21.15.102:0/1027113649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b4101720 msgr2=0x7fc9b4197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9baa8b640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b4101720 0x7fc9b4197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9baa8b640 1 -- 172.21.15.102:0/1027113649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9b41020e0 msgr2=0x7fc9b4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9baa8b640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9b41020e0 0x7fc9b4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9baa8b640 1 -- 172.21.15.102:0/1027113649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b419cf00 con 0x7fc9b4107730 2024-04-24T23:28:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9ba28a640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b4101720 0x7fc9b4197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9baa8b640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b4107730 0x7fc9b419c7d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005cc0 tx=0x7fc9ac003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9ab7fe640 1 -- 172.21.15.102:0/1027113649 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac004450 con 0x7fc9b4107730 2024-04-24T23:28:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b419d190 con 0x7fc9b4107730 2024-04-24T23:28:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9ab7fe640 1 -- 172.21.15.102:0/1027113649 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9ac031c90 con 0x7fc9b4107730 2024-04-24T23:28:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9ab7fe640 1 -- 172.21.15.102:0/1027113649 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac03b400 con 0x7fc9b4107730 2024-04-24T23:28:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.061+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b41a4a80 con 0x7fc9b4107730 2024-04-24T23:28:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.062+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b4068560 con 0x7fc9b4107730 2024-04-24T23:28:08.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.064+0000 7fc9ab7fe640 1 -- 172.21.15.102:0/1027113649 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fc9ac004680 con 0x7fc9b4107730 2024-04-24T23:28:08.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.064+0000 7fc9ab7fe640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc9a00a2d00 0x7fc9a00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.064+0000 7fc9ba28a640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc9a00a2d00 0x7fc9a00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.064+0000 7fc9ab7fe640 1 -- 172.21.15.102:0/1027113649 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7fc9ac02e030 con 0x7fc9b4107730 2024-04-24T23:28:08.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.065+0000 7fc9ba28a640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc9a00a2d00 0x7fc9a00a51c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0005d70 tx=0x7fc9b0005d00 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:08.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.068+0000 7fc9ab7fe640 1 -- 172.21.15.102:0/1027113649 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc9ac0bd7b0 con 0x7fc9b4107730 2024-04-24T23:28:08.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.262+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9b402d170 con 0x7fc9b4107730 2024-04-24T23:28:08.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.262+0000 7fc9ab7fe640 1 -- 172.21.15.102:0/1027113649 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) ==== 81+0+961 (secure 0 0 0) 0x7fc9ac0bd0b0 con 0x7fc9b4107730 2024-04-24T23:28:08.263 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2090298312 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/835175442 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3958169470 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/3611055373 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/3017038970 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3702328937 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1394402685 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/4171285045 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/4149172465 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1812533171 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/3017038970 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2445922755 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1700818231 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/2363268578 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/3611055373 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:08.264 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/2363268578 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:08.265 INFO:teuthology.orchestra.run.smithi102.stderr:listed 16 entries 2024-04-24T23:28:08.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc9a00a2d00 msgr2=0x7fc9a00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc9a00a2d00 0x7fc9a00a51c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0005d70 tx=0x7fc9b0005d00 comp rx=0 tx=0).stop 2024-04-24T23:28:08.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b4107730 msgr2=0x7fc9b419c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b4107730 0x7fc9b419c7d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005cc0 tx=0x7fc9ac003c20 comp rx=0 tx=0).stop 2024-04-24T23:28:08.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 shutdown_connections 2024-04-24T23:28:08.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc9a00a2d00 0x7fc9a00a51c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b4107730 0x7fc9b419c7d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9b41020e0 0x7fc9b4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 --2- 172.21.15.102:0/1027113649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b4101720 0x7fc9b4197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 >> 172.21.15.102:0/1027113649 conn(0x7fc9b4078b40 msgr2=0x7fc9b4104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:08.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 shutdown_connections 2024-04-24T23:28:08.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.264+0000 7fc9bc4ee640 1 -- 172.21.15.102:0/1027113649 wait complete. 2024-04-24T23:28:08.275 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:08 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3527729986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:28:08.275 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:08 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4136366028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:28:08.275 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:08 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/2907783796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:28:08.276 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-24T23:28:08.276 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:28:08.292 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:28:08.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.429+0000 7fc07cd89640 1 -- 172.21.15.102:0/3884661793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc078100050 msgr2=0x7fc078100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.429+0000 7fc07cd89640 1 --2- 172.21.15.102:0/3884661793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc078100050 0x7fc078100430 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc07000aa10 tx=0x7fc07002b720 comp rx=0 tx=0).stop 2024-04-24T23:28:08.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.429+0000 7fc07cd89640 1 -- 172.21.15.102:0/3884661793 shutdown_connections 2024-04-24T23:28:08.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.429+0000 7fc07cd89640 1 --2- 172.21.15.102:0/3884661793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0781050d0 0x7fc07810cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.429+0000 7fc07cd89640 1 --2- 172.21.15.102:0/3884661793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc078100a00 0x7fc0781049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.429+0000 7fc07cd89640 1 --2- 172.21.15.102:0/3884661793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc078100050 0x7fc078100430 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.429+0000 7fc07cd89640 1 -- 172.21.15.102:0/3884661793 >> 172.21.15.102:0/3884661793 conn(0x7fc078076c70 msgr2=0x7fc0780fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:08.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.429+0000 7fc07cd89640 1 -- 172.21.15.102:0/3884661793 shutdown_connections 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.429+0000 7fc07cd89640 1 -- 172.21.15.102:0/3884661793 wait complete. 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc07cd89640 1 Processor -- start 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc07cd89640 1 -- start start 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc07cd89640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078100050 0x7fc078197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc07cd89640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0781050d0 0x7fc078198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc07cd89640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc07819c790 0x7fc07819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc07cd89640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc07810f7b0 con 0x7fc07819c790 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc07cd89640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc07810f630 con 0x7fc0781050d0 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc07cd89640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc07810f930 con 0x7fc078100050 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc076d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc07819c790 0x7fc07819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc076d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc07819c790 0x7fc07819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53822/0 (socket says 172.21.15.102:53822) 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc076d9d640 1 -- 172.21.15.102:0/921835749 learned_addr learned my addr 172.21.15.102:0/921835749 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc075d9b640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0781050d0 0x7fc078198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc076d9d640 1 -- 172.21.15.102:0/921835749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078100050 msgr2=0x7fc078197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc076d9d640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078100050 0x7fc078197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc076d9d640 1 -- 172.21.15.102:0/921835749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0781050d0 msgr2=0x7fc078198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc076d9d640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0781050d0 0x7fc078198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc076d9d640 1 -- 172.21.15.102:0/921835749 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07819d280 con 0x7fc07819c790 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc075d9b640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0781050d0 0x7fc078198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.430+0000 7fc076d9d640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc07819c790 0x7fc07819cc40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc06800c970 tx=0x7fc06800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.431+0000 7fc0677fe640 1 -- 172.21.15.102:0/921835749 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068007bb0 con 0x7fc07819c790 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.431+0000 7fc0677fe640 1 -- 172.21.15.102:0/921835749 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc068007d50 con 0x7fc07819c790 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.431+0000 7fc0677fe640 1 -- 172.21.15.102:0/921835749 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068018700 con 0x7fc07819c790 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.431+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07819d570 con 0x7fc07819c790 2024-04-24T23:28:08.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.431+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0781a4d10 con 0x7fc07819c790 2024-04-24T23:28:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.432+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0780689b0 con 0x7fc07819c790 2024-04-24T23:28:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.433+0000 7fc0677fe640 1 -- 172.21.15.102:0/921835749 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fc06800fa20 con 0x7fc07819c790 2024-04-24T23:28:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.434+0000 7fc0677fe640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc0600a2d00 0x7fc0600a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.434+0000 7fc07659c640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc0600a2d00 0x7fc0600a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.434+0000 7fc0677fe640 1 -- 172.21.15.102:0/921835749 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7fc0680cf700 con 0x7fc07819c790 2024-04-24T23:28:08.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.434+0000 7fc07659c640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc0600a2d00 0x7fc0600a51c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc07000a7d0 tx=0x7fc070030510 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:08.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.437+0000 7fc0677fe640 1 -- 172.21.15.102:0/921835749 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc068014030 con 0x7fc07819c790 2024-04-24T23:28:08.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:08 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3527729986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:28:08.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:08 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4136366028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:28:08.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:08 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/2907783796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:28:08.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:08 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3527729986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:28:08.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:08 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4136366028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:28:08.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:08 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/2907783796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:28:08.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.616+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc078199220 con 0x7fc07819c790 2024-04-24T23:28:08.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.618+0000 7fc0677fe640 1 -- 172.21.15.102:0/921835749 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+23244 (secure 0 0 0) 0x7fc078199220 con 0x7fc07819c790 2024-04-24T23:28:08.619 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:28:08.619 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":107,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:27:20.075691+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/2090298312":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/835175442":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/3611055373":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/4149172465":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/1812533171":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/3611055373":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:28:08.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc0600a2d00 msgr2=0x7fc0600a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc0600a2d00 0x7fc0600a51c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc07000a7d0 tx=0x7fc070030510 comp rx=0 tx=0).stop 2024-04-24T23:28:08.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc07819c790 msgr2=0x7fc07819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc07819c790 0x7fc07819cc40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc06800c970 tx=0x7fc06800ce30 comp rx=0 tx=0).stop 2024-04-24T23:28:08.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 shutdown_connections 2024-04-24T23:28:08.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fc0600a2d00 0x7fc0600a51c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc07819c790 0x7fc07819cc40 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0781050d0 0x7fc078198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 --2- 172.21.15.102:0/921835749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078100050 0x7fc078197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 >> 172.21.15.102:0/921835749 conn(0x7fc078076c70 msgr2=0x7fc078103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:08.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 shutdown_connections 2024-04-24T23:28:08.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.620+0000 7fc07cd89640 1 -- 172.21.15.102:0/921835749 wait complete. 2024-04-24T23:28:08.632 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:06:16.456403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:10:58.730586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:10:59.855279+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:28:08.632 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:28:08.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.732+0000 7f8f18ab3640 1 -- 172.21.15.102:0/3617725739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 msgr2=0x7f8f1410e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.732+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/3617725739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 0x7f8f1410e650 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009cd0 tx=0x7f8f0c02a8d0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.732+0000 7f8f18ab3640 1 -- 172.21.15.102:0/3617725739 shutdown_connections 2024-04-24T23:28:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.732+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/3617725739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 0x7f8f1410e650 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.732+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/3617725739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f141008d0 0x7f8f14106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.732+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/3617725739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f140fff20 0x7f8f14100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.732+0000 7f8f18ab3640 1 -- 172.21.15.102:0/3617725739 >> 172.21.15.102:0/3617725739 conn(0x7f8f14076c40 msgr2=0x7f8f140fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.732+0000 7f8f18ab3640 1 -- 172.21.15.102:0/3617725739 shutdown_connections 2024-04-24T23:28:08.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.732+0000 7f8f18ab3640 1 -- 172.21.15.102:0/3617725739 wait complete. 2024-04-24T23:28:08.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f18ab3640 1 Processor -- start 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f18ab3640 1 -- start start 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f18ab3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f140fff20 0x7f8f14197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f18ab3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 0x7f8f14198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f18ab3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f1419c860 0x7f8f1419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f18ab3640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8f141119b0 con 0x7f8f14106a40 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f18ab3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14111830 con 0x7f8f140fff20 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f18ab3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14111b30 con 0x7f8f1419c860 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f09d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 0x7f8f14198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f09d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 0x7f8f14198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53832/0 (socket says 172.21.15.102:53832) 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.733+0000 7f8f09d9b640 1 -- 172.21.15.102:0/255870616 learned_addr learned my addr 172.21.15.102:0/255870616 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f1259c640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f140fff20 0x7f8f14197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f12d9d640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f1419c860 0x7f8f1419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f09d9b640 1 -- 172.21.15.102:0/255870616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f1419c860 msgr2=0x7f8f1419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f09d9b640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f1419c860 0x7f8f1419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f09d9b640 1 -- 172.21.15.102:0/255870616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f140fff20 msgr2=0x7f8f14197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f09d9b640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f140fff20 0x7f8f14197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f09d9b640 1 -- 172.21.15.102:0/255870616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1419d350 con 0x7f8f14106a40 2024-04-24T23:28:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f1259c640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f140fff20 0x7f8f14197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f09d9b640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 0x7f8f14198530 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00d770 tx=0x7f8efc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:08.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f0bfff640 1 -- 172.21.15.102:0/255870616 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc017070 con 0x7f8f14106a40 2024-04-24T23:28:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f0bfff640 1 -- 172.21.15.102:0/255870616 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8efc009c30 con 0x7f8f14106a40 2024-04-24T23:28:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1419d640 con 0x7f8f14106a40 2024-04-24T23:28:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f0bfff640 1 -- 172.21.15.102:0/255870616 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00f620 con 0x7f8f14106a40 2024-04-24T23:28:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.734+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141a4e30 con 0x7f8f14106a40 2024-04-24T23:28:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.735+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f140689b0 con 0x7f8f14106a40 2024-04-24T23:28:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.736+0000 7f8f0bfff640 1 -- 172.21.15.102:0/255870616 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8efc01a070 con 0x7f8f14106a40 2024-04-24T23:28:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.737+0000 7f8f0bfff640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8f000a2d00 0x7f8f000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.737+0000 7f8f1259c640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8f000a2d00 0x7f8f000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:08.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.737+0000 7f8f0bfff640 1 -- 172.21.15.102:0/255870616 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7f8efc0cf000 con 0x7f8f14106a40 2024-04-24T23:28:08.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.737+0000 7f8f1259c640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8f000a2d00 0x7f8f000a51c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009b40 tx=0x7f8f04005e50 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:08.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.740+0000 7f8f0bfff640 1 -- 172.21.15.102:0/255870616 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8efc009de0 con 0x7f8f14106a40 2024-04-24T23:28:08.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.915+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f141008d0 con 0x7f8f14106a40 2024-04-24T23:28:08.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.915+0000 7f8f0bfff640 1 -- 172.21.15.102:0/255870616 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7f8efc09bbe0 con 0x7f8f14106a40 2024-04-24T23:28:08.917 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-04-24T23:28:08.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8f000a2d00 msgr2=0x7f8f000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8f000a2d00 0x7f8f000a51c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009b40 tx=0x7f8f04005e50 comp rx=0 tx=0).stop 2024-04-24T23:28:08.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 msgr2=0x7f8f14198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:08.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 0x7f8f14198530 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00d770 tx=0x7f8efc00dc30 comp rx=0 tx=0).stop 2024-04-24T23:28:08.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 shutdown_connections 2024-04-24T23:28:08.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f1419c860 0x7f8f1419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f8f000a2d00 0x7f8f000a51c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f14106a40 0x7f8f14198530 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 --2- 172.21.15.102:0/255870616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f140fff20 0x7f8f14197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:08.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.917+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 >> 172.21.15.102:0/255870616 conn(0x7f8f14076c40 msgr2=0x7f8f140fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:08.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.918+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 shutdown_connections 2024-04-24T23:28:08.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:08.918+0000 7f8f18ab3640 1 -- 172.21.15.102:0/255870616 wait complete. 2024-04-24T23:28:08.929 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:28:09.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 -- 172.21.15.102:0/2772029533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5198101710 msgr2=0x7f5198101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 --2- 172.21.15.102:0/2772029533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5198101710 0x7f5198101b90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5180009930 tx=0x7f518002a420 comp rx=0 tx=0).stop 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 -- 172.21.15.102:0/2772029533 shutdown_connections 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 --2- 172.21.15.102:0/2772029533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51981020d0 0x7f519810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 --2- 172.21.15.102:0/2772029533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5198101710 0x7f5198101b90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 --2- 172.21.15.102:0/2772029533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198107720 0x7f5198107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 -- 172.21.15.102:0/2772029533 >> 172.21.15.102:0/2772029533 conn(0x7f5198078b90 msgr2=0x7f5198078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 -- 172.21.15.102:0/2772029533 shutdown_connections 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 -- 172.21.15.102:0/2772029533 wait complete. 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.029+0000 7f519d31b640 1 Processor -- start 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f519d31b640 1 -- start start 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f519d31b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198101710 0x7f5198197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f519d31b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51981020d0 0x7f51981984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f519d31b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5198107720 0x7f519819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f519d31b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51981119e0 con 0x7f51981020d0 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f519d31b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5198111860 con 0x7f5198101710 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f519d31b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5198111b60 con 0x7f5198107720 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f518ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51981020d0 0x7f51981984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f518ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51981020d0 0x7f51981984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53848/0 (socket says 172.21.15.102:53848) 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f518ffff640 1 -- 172.21.15.102:0/225386896 learned_addr learned my addr 172.21.15.102:0/225386896 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f5196ffd640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198101710 0x7f5198197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f518ffff640 1 -- 172.21.15.102:0/225386896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5198107720 msgr2=0x7f519819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f51977fe640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5198107720 0x7f519819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f518ffff640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5198107720 0x7f519819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f518ffff640 1 -- 172.21.15.102:0/225386896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198101710 msgr2=0x7f5198197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f518ffff640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198101710 0x7f5198197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f518ffff640 1 -- 172.21.15.102:0/225386896 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519819cf40 con 0x7f51981020d0 2024-04-24T23:28:09.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f518ffff640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51981020d0 0x7f51981984e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5180005940 tx=0x7f51800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f5194ff9640 1 -- 172.21.15.102:0/225386896 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518003f030 con 0x7f51981020d0 2024-04-24T23:28:09.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f5194ff9640 1 -- 172.21.15.102:0/225386896 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5180003cb0 con 0x7f51981020d0 2024-04-24T23:28:09.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519819d1d0 con 0x7f51981020d0 2024-04-24T23:28:09.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.030+0000 7f5194ff9640 1 -- 172.21.15.102:0/225386896 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180035070 con 0x7f51981020d0 2024-04-24T23:28:09.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.031+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51981a4ac0 con 0x7f51981020d0 2024-04-24T23:28:09.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.031+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51980689b0 con 0x7f51981020d0 2024-04-24T23:28:09.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.032+0000 7f5194ff9640 1 -- 172.21.15.102:0/225386896 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f5180042030 con 0x7f51981020d0 2024-04-24T23:28:09.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.033+0000 7f5194ff9640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f51840a2d00 0x7f51840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.033+0000 7f5196ffd640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f51840a2d00 0x7f51840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.033+0000 7f5194ff9640 1 -- 172.21.15.102:0/225386896 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7f51800efc80 con 0x7f51981020d0 2024-04-24T23:28:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.034+0000 7f5196ffd640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f51840a2d00 0x7f51840a51c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f518800abe0 tx=0x7f518800a2d0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.037+0000 7f5194ff9640 1 -- 172.21.15.102:0/225386896 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f51800bc860 con 0x7f51981020d0 2024-04-24T23:28:09.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.212+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5198109710 con 0x7f51981020d0 2024-04-24T23:28:09.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.212+0000 7f5194ff9640 1 -- 172.21.15.102:0/225386896 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f51800bc160 con 0x7f51981020d0 2024-04-24T23:28:09.214 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-04-24T23:28:09.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f51840a2d00 msgr2=0x7f51840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f51840a2d00 0x7f51840a51c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f518800abe0 tx=0x7f518800a2d0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51981020d0 msgr2=0x7f51981984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51981020d0 0x7f51981984e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5180005940 tx=0x7f51800042f0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 shutdown_connections 2024-04-24T23:28:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5198107720 0x7f519819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f51840a2d00 0x7f51840a51c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51981020d0 0x7f51981984e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 --2- 172.21.15.102:0/225386896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198101710 0x7f5198197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 >> 172.21.15.102:0/225386896 conn(0x7f5198078b90 msgr2=0x7f51980ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 shutdown_connections 2024-04-24T23:28:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.214+0000 7f519d31b640 1 -- 172.21.15.102:0/225386896 wait complete. 2024-04-24T23:28:09.225 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:09 smithi102 ceph-mon[29216]: pgmap v472: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T23:28:09.225 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:09 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1027113649' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:28:09.225 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:09 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/921835749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:09.225 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:09 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/255870616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:28:09.226 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:28:09.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.325+0000 7fd665212640 1 -- 172.21.15.102:0/966627988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6601066b0 msgr2=0x7fd660106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.325+0000 7fd665212640 1 --2- 172.21.15.102:0/966627988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6601066b0 0x7fd660106a90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd650009930 tx=0x7fd65002a420 comp rx=0 tx=0).stop 2024-04-24T23:28:09.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.325+0000 7fd665212640 1 -- 172.21.15.102:0/966627988 shutdown_connections 2024-04-24T23:28:09.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.325+0000 7fd665212640 1 --2- 172.21.15.102:0/966627988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd660100fd0 0x7fd66010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.325+0000 7fd665212640 1 --2- 172.21.15.102:0/966627988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd660100610 0x7fd660100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.325+0000 7fd665212640 1 --2- 172.21.15.102:0/966627988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6601066b0 0x7fd660106a90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.325+0000 7fd665212640 1 -- 172.21.15.102:0/966627988 >> 172.21.15.102:0/966627988 conn(0x7fd660077740 msgr2=0x7fd660077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.325+0000 7fd665212640 1 -- 172.21.15.102:0/966627988 shutdown_connections 2024-04-24T23:28:09.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.325+0000 7fd665212640 1 -- 172.21.15.102:0/966627988 wait complete. 2024-04-24T23:28:09.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd665212640 1 Processor -- start 2024-04-24T23:28:09.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd665212640 1 -- start start 2024-04-24T23:28:09.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd665212640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd660100610 0x7fd660198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd665212640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd660100fd0 0x7fd660199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd665212640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6601066b0 0x7fd6601a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd665212640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601108a0 con 0x7fd660100fd0 2024-04-24T23:28:09.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd665212640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd660110720 con 0x7fd6601066b0 2024-04-24T23:28:09.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd665212640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd660110a20 con 0x7fd660100610 2024-04-24T23:28:09.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd657fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd660100fd0 0x7fd660199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd657fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd660100fd0 0x7fd660199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53864/0 (socket says 172.21.15.102:53864) 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd657fff640 1 -- 172.21.15.102:0/104498161 learned_addr learned my addr 172.21.15.102:0/104498161 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd65f59e640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6601066b0 0x7fd6601a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd657fff640 1 -- 172.21.15.102:0/104498161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd660100610 msgr2=0x7fd660198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd657fff640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd660100610 0x7fd660198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd657fff640 1 -- 172.21.15.102:0/104498161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6601066b0 msgr2=0x7fd6601a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd657fff640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6601066b0 0x7fd6601a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.326+0000 7fd657fff640 1 -- 172.21.15.102:0/104498161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6601a4280 con 0x7fd660100fd0 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.327+0000 7fd657fff640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd660100fd0 0x7fd660199200 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c890 tx=0x7fd64800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.327+0000 7fd65cd99640 1 -- 172.21.15.102:0/104498161 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648004220 con 0x7fd660100fd0 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.327+0000 7fd65cd99640 1 -- 172.21.15.102:0/104498161 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6480043c0 con 0x7fd660100fd0 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.327+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6601a44b0 con 0x7fd660100fd0 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.327+0000 7fd65cd99640 1 -- 172.21.15.102:0/104498161 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648004a00 con 0x7fd660100fd0 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.327+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6601a49d0 con 0x7fd660100fd0 2024-04-24T23:28:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.328+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd660068560 con 0x7fd660100fd0 2024-04-24T23:28:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.329+0000 7fd65cd99640 1 -- 172.21.15.102:0/104498161 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fd648004ba0 con 0x7fd660100fd0 2024-04-24T23:28:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.329+0000 7fd65cd99640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd64c0a2d00 0x7fd64c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.329+0000 7fd65ed9d640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd64c0a2d00 0x7fd64c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.329+0000 7fd65cd99640 1 -- 172.21.15.102:0/104498161 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7fd6480cf200 con 0x7fd660100fd0 2024-04-24T23:28:09.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.330+0000 7fd65ed9d640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd64c0a2d00 0x7fd64c0a51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd6500096c0 tx=0x7fd650031400 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.334+0000 7fd65cd99640 1 -- 172.21.15.102:0/104498161 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd648014030 con 0x7fd660100fd0 2024-04-24T23:28:09.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.507+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd6601085f0 con 0x7fd660100fd0 2024-04-24T23:28:09.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.507+0000 7fd65cd99640 1 -- 172.21.15.102:0/104498161 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7fd64809bde0 con 0x7fd660100fd0 2024-04-24T23:28:09.508 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-04-24T23:28:09.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd64c0a2d00 msgr2=0x7fd64c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd64c0a2d00 0x7fd64c0a51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd6500096c0 tx=0x7fd650031400 comp rx=0 tx=0).stop 2024-04-24T23:28:09.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd660100fd0 msgr2=0x7fd660199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd660100fd0 0x7fd660199200 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c890 tx=0x7fd64800cd50 comp rx=0 tx=0).stop 2024-04-24T23:28:09.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 shutdown_connections 2024-04-24T23:28:09.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6601066b0 0x7fd6601a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fd64c0a2d00 0x7fd64c0a51c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd660100fd0 0x7fd660199200 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 --2- 172.21.15.102:0/104498161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd660100610 0x7fd660198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 >> 172.21.15.102:0/104498161 conn(0x7fd660077740 msgr2=0x7fd6600ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 shutdown_connections 2024-04-24T23:28:09.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.509+0000 7fd665212640 1 -- 172.21.15.102:0/104498161 wait complete. 2024-04-24T23:28:09.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:09 smithi183 ceph-mon[33137]: pgmap v472: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T23:28:09.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:09 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1027113649' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:28:09.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:09 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/921835749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:09.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:09 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/255870616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:28:09.521 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:28:09.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:09 smithi161 ceph-mon[32709]: pgmap v472: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T23:28:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:09 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1027113649' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:28:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:09 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/921835749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:09.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:09 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/255870616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.618+0000 7f840e6da640 1 -- 172.21.15.102:0/1172685687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84081032e0 msgr2=0x7f840810b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.618+0000 7f840e6da640 1 --2- 172.21.15.102:0/1172685687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84081032e0 0x7f840810b7c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009900 tx=0x7f83fc02a510 comp rx=0 tx=0).stop 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.618+0000 7f840e6da640 1 -- 172.21.15.102:0/1172685687 shutdown_connections 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.618+0000 7f840e6da640 1 --2- 172.21.15.102:0/1172685687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f840810bd00 0x7f840810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.618+0000 7f840e6da640 1 --2- 172.21.15.102:0/1172685687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84081032e0 0x7f840810b7c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.618+0000 7f840e6da640 1 --2- 172.21.15.102:0/1172685687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84081029c0 0x7f8408102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.618+0000 7f840e6da640 1 -- 172.21.15.102:0/1172685687 >> 172.21.15.102:0/1172685687 conn(0x7f8408077740 msgr2=0x7f8408077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.618+0000 7f840e6da640 1 -- 172.21.15.102:0/1172685687 shutdown_connections 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.618+0000 7f840e6da640 1 -- 172.21.15.102:0/1172685687 wait complete. 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840e6da640 1 Processor -- start 2024-04-24T23:28:09.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840e6da640 1 -- start start 2024-04-24T23:28:09.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840e6da640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84081029c0 0x7f8408197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840e6da640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84081032e0 0x7f8408198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840e6da640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f840810bd00 0x7f840819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840e6da640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f84081119b0 con 0x7f840810bd00 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840e6da640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8408111830 con 0x7f84081029c0 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840e6da640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8408111b30 con 0x7f84081032e0 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840cc77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f840810bd00 0x7f840819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840cc77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f840810bd00 0x7f840819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53880/0 (socket says 172.21.15.102:53880) 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.619+0000 7f840cc77640 1 -- 172.21.15.102:0/1151630413 learned_addr learned my addr 172.21.15.102:0/1151630413 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f84077fe640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84081032e0 0x7f8408198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f8407fff640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84081029c0 0x7f8408197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f840cc77640 1 -- 172.21.15.102:0/1151630413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84081032e0 msgr2=0x7f8408198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f840cc77640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84081032e0 0x7f8408198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f840cc77640 1 -- 172.21.15.102:0/1151630413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84081029c0 msgr2=0x7f8408197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f840cc77640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84081029c0 0x7f8408197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f840cc77640 1 -- 172.21.15.102:0/1151630413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840819cec0 con 0x7f840810bd00 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f8407fff640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84081029c0 0x7f8408197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:09.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f84077fe640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84081032e0 0x7f8408198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f840cc77640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f840810bd00 0x7f840819c790 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f840000cce0 tx=0x7f840000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f84057fa640 1 -- 172.21.15.102:0/1151630413 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840000f450 con 0x7f840810bd00 2024-04-24T23:28:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f84057fa640 1 -- 172.21.15.102:0/1151630413 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f840000faa0 con 0x7f840810bd00 2024-04-24T23:28:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840819d1b0 con 0x7f840810bd00 2024-04-24T23:28:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f84057fa640 1 -- 172.21.15.102:0/1151630413 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400018b70 con 0x7f840810bd00 2024-04-24T23:28:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.620+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081a4a40 con 0x7f840810bd00 2024-04-24T23:28:09.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.621+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8408068560 con 0x7f840810bd00 2024-04-24T23:28:09.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.622+0000 7f84057fa640 1 -- 172.21.15.102:0/1151630413 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f8400018d10 con 0x7f840810bd00 2024-04-24T23:28:09.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.623+0000 7f84057fa640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f83f00a2d00 0x7f83f00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:09.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.623+0000 7f8407fff640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f83f00a2d00 0x7f83f00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:09.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.623+0000 7f84057fa640 1 -- 172.21.15.102:0/1151630413 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8579+0+0 (secure 0 0 0) 0x7f84000cf910 con 0x7f840810bd00 2024-04-24T23:28:09.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.623+0000 7f8407fff640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f83f00a2d00 0x7f83f00a51c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83f8005d00 tx=0x7f83f8009550 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:09.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.627+0000 7f84057fa640 1 -- 172.21.15.102:0/1151630413 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8400014030 con 0x7f840810bd00 2024-04-24T23:28:09.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.816+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f84081068f0 con 0x7f840810bd00 2024-04-24T23:28:09.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.816+0000 7f84057fa640 1 -- 172.21.15.102:0/1151630413 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) ==== 81+0+961 (secure 0 0 0) 0x7f840009c4f0 con 0x7f840810bd00 2024-04-24T23:28:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2090298312 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:09.817 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/835175442 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3958169470 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/3611055373 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/3017038970 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3702328937 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1394402685 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/4171285045 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/4149172465 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1812533171 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/3017038970 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2445922755 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1700818231 2024-04-25T23:03:48.134719+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/2363268578 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:09.818 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/3611055373 2024-04-25T23:13:01.525573+0000 2024-04-24T23:28:09.819 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/2363268578 2024-04-25T23:03:21.040554+0000 2024-04-24T23:28:09.819 INFO:teuthology.orchestra.run.smithi102.stderr:listed 16 entries 2024-04-24T23:28:09.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f83f00a2d00 msgr2=0x7f83f00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f83f00a2d00 0x7f83f00a51c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83f8005d00 tx=0x7f83f8009550 comp rx=0 tx=0).stop 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f840810bd00 msgr2=0x7f840819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f840810bd00 0x7f840819c790 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f840000cce0 tx=0x7f840000a620 comp rx=0 tx=0).stop 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 shutdown_connections 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f83f00a2d00 0x7f83f00a51c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f840810bd00 0x7f840819c790 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84081032e0 0x7f8408198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 --2- 172.21.15.102:0/1151630413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84081029c0 0x7f8408197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 >> 172.21.15.102:0/1151630413 conn(0x7f8408077740 msgr2=0x7f84080fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 shutdown_connections 2024-04-24T23:28:09.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:09.818+0000 7f840e6da640 1 -- 172.21.15.102:0/1151630413 wait complete. 2024-04-24T23:28:09.830 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi102.front.sepia.ceph.com... 2024-04-24T23:28:09.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:09.834 DEBUG:teuthology.orchestra.run.smithi102:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T23:28:09.879 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:28:09.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:ceph-fuse[81764]: fuse finished with error 0 and tester_r 0 2024-04-24T23:28:10.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:10 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/225386896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:28:10.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:10 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/104498161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:28:10.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:10 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1151630413' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:28:10.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:10 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/225386896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:28:10.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:10 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/104498161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:28:10.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:10 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1151630413' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:28:10.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:10 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/225386896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:28:10.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:10 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/104498161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:28:10.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:10 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1151630413' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:28:11.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:11 smithi183 ceph-mon[33137]: pgmap v473: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-24T23:28:11.570 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:11 smithi102 ceph-mon[29216]: pgmap v473: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-24T23:28:11.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:11 smithi161 ceph-mon[32709]: pgmap v473: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-24T23:28:12.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:12 smithi183 ceph-mon[33137]: osdmap e108: 12 total, 12 up, 12 in 2024-04-24T23:28:12.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:12 smithi102 ceph-mon[29216]: osdmap e108: 12 total, 12 up, 12 in 2024-04-24T23:28:12.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:12 smithi161 ceph-mon[32709]: osdmap e108: 12 total, 12 up, 12 in 2024-04-24T23:28:13.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:13 smithi183 ceph-mon[33137]: pgmap v475: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T23:28:13.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:13 smithi102 ceph-mon[29216]: pgmap v475: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T23:28:13.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:13 smithi161 ceph-mon[32709]: pgmap v475: 97 pgs: 97 active+clean; 18 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T23:28:15.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:15 smithi183 ceph-mon[33137]: pgmap v476: 97 pgs: 21 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 65 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-04-24T23:28:15.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:15 smithi102 ceph-mon[29216]: pgmap v476: 97 pgs: 21 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 65 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-04-24T23:28:15.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:15 smithi161 ceph-mon[32709]: pgmap v476: 97 pgs: 21 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 65 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-04-24T23:28:15.980 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:28:15.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:15.981 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:28:16.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:16.004 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-04-24T23:28:16.071 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:28:16.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:16.071 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:28:16.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:28:16.142 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link delete ceph-brx 2024-04-24T23:28:16.239 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T23:28:16.250 INFO:tasks.cephadm:Teardown begin 2024-04-24T23:28:16.251 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:28:16.251 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:16.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.353+0000 7f5bb5f29640 1 -- 172.21.15.102:0/2417730334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01032c0 msgr2=0x7f5bb010b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.353+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/2417730334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01032c0 0x7f5bb010b7a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009900 tx=0x7f5ba402a480 comp rx=0 tx=0).stop 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.354+0000 7f5bb5f29640 1 -- 172.21.15.102:0/2417730334 shutdown_connections 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.354+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/2417730334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb010bd70 0x7f5bb010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.354+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/2417730334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01032c0 0x7f5bb010b7a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.354+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/2417730334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb01029a0 0x7f5bb0102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.354+0000 7f5bb5f29640 1 -- 172.21.15.102:0/2417730334 >> 172.21.15.102:0/2417730334 conn(0x7f5bb00769c0 msgr2=0x7f5bb0076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.354+0000 7f5bb5f29640 1 -- 172.21.15.102:0/2417730334 shutdown_connections 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.354+0000 7f5bb5f29640 1 -- 172.21.15.102:0/2417730334 wait complete. 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.354+0000 7f5bb5f29640 1 Processor -- start 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.354+0000 7f5bb5f29640 1 -- start start 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bb5f29640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01029a0 0x7f5bb0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bb5f29640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb01032c0 0x7f5bb0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bb5f29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb010bd70 0x7f5bb019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bb5f29640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0111a10 con 0x7f5bb01029a0 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bb5f29640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0111890 con 0x7f5bb01032c0 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bb5f29640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0111b90 con 0x7f5bb010bd70 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01029a0 0x7f5bb0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01029a0 0x7f5bb0198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53900/0 (socket says 172.21.15.102:53900) 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baf7fe640 1 -- 172.21.15.102:0/1436012585 learned_addr learned my addr 172.21.15.102:0/1436012585 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baeffd640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb01032c0 0x7f5bb0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baffff640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb010bd70 0x7f5bb019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baf7fe640 1 -- 172.21.15.102:0/1436012585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb010bd70 msgr2=0x7f5bb019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baf7fe640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb010bd70 0x7f5bb019c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baf7fe640 1 -- 172.21.15.102:0/1436012585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb01032c0 msgr2=0x7f5bb0198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baf7fe640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb01032c0 0x7f5bb0198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baf7fe640 1 -- 172.21.15.102:0/1436012585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb019cff0 con 0x7f5bb01029a0 2024-04-24T23:28:16.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baeffd640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb01032c0 0x7f5bb0198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5baf7fe640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01029a0 0x7f5bb0198000 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000b4a0 tx=0x7f5ba000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bacff9640 1 -- 172.21.15.102:0/1436012585 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0014070 con 0x7f5bb01029a0 2024-04-24T23:28:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bacff9640 1 -- 172.21.15.102:0/1436012585 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba0010ac0 con 0x7f5bb01029a0 2024-04-24T23:28:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb019d2e0 con 0x7f5bb01029a0 2024-04-24T23:28:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bacff9640 1 -- 172.21.15.102:0/1436012585 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0017c30 con 0x7f5bb01029a0 2024-04-24T23:28:16.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.355+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb01a4b70 con 0x7f5bb01029a0 2024-04-24T23:28:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.356+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb006a5c0 con 0x7f5bb01029a0 2024-04-24T23:28:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.358+0000 7f5bacff9640 1 -- 172.21.15.102:0/1436012585 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f5ba0017070 con 0x7f5bb01029a0 2024-04-24T23:28:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.359+0000 7f5bacff9640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5b980a2d00 0x7f5b980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.359+0000 7f5baeffd640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5b980a2d00 0x7f5b980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.359+0000 7f5bacff9640 1 -- 172.21.15.102:0/1436012585 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8623+0+0 (secure 0 0 0) 0x7f5ba00ceb20 con 0x7f5bb01029a0 2024-04-24T23:28:16.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.359+0000 7f5baeffd640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5b980a2d00 0x7f5b980a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4000c00 tx=0x7f5ba4031400 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.362+0000 7f5bacff9640 1 -- 172.21.15.102:0/1436012585 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5ba00cee20 con 0x7f5bb01029a0 2024-04-24T23:28:16.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.581+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bb00689b0 con 0x7f5bb01029a0 2024-04-24T23:28:16.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.582+0000 7f5bacff9640 1 -- 172.21.15.102:0/1436012585 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11887 (secure 0 0 0) 0x7f5ba009b7f0 con 0x7f5bb01029a0 2024-04-24T23:28:16.583 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:28:16.583 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:11:00.643113+0000","modified":"2024-04-24T23:27:01.805407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/1258581930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1258581930},{"type":"v1","addr":"172.21.15.161:6839","nonce":1258581930}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24473,24497,24472]},"id":1}]} 2024-04-24T23:28:16.583 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 29 2024-04-24T23:28:16.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5b980a2d00 msgr2=0x7f5b980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5b980a2d00 0x7f5b980a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4000c00 tx=0x7f5ba4031400 comp rx=0 tx=0).stop 2024-04-24T23:28:16.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01029a0 msgr2=0x7f5bb0198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01029a0 0x7f5bb0198000 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000b4a0 tx=0x7f5ba000b960 comp rx=0 tx=0).stop 2024-04-24T23:28:16.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 shutdown_connections 2024-04-24T23:28:16.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bb010bd70 0x7f5bb019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bb01032c0 0x7f5bb0198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f5b980a2d00 0x7f5b980a51c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 --2- 172.21.15.102:0/1436012585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5bb01029a0 0x7f5bb0198000 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 >> 172.21.15.102:0/1436012585 conn(0x7f5bb00769c0 msgr2=0x7f5bb00fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:16.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 shutdown_connections 2024-04-24T23:28:16.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.584+0000 7f5bb5f29640 1 -- 172.21.15.102:0/1436012585 wait complete. 2024-04-24T23:28:16.597 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.694+0000 7fcf02c7b640 1 -- 172.21.15.102:0/175407186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc100fe0 msgr2=0x7fcefc10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.694+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/175407186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc100fe0 0x7fcefc10d4f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009930 tx=0x7fcee402a420 comp rx=0 tx=0).stop 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.694+0000 7fcf02c7b640 1 -- 172.21.15.102:0/175407186 shutdown_connections 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.694+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/175407186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc100fe0 0x7fcefc10d4f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.694+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/175407186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcefc100620 0x7fcefc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.694+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/175407186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcefc106630 0x7fcefc106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.694+0000 7fcf02c7b640 1 -- 172.21.15.102:0/175407186 >> 172.21.15.102:0/175407186 conn(0x7fcefc077740 msgr2=0x7fcefc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 -- 172.21.15.102:0/175407186 shutdown_connections 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 -- 172.21.15.102:0/175407186 wait complete. 2024-04-24T23:28:16.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 Processor -- start 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 -- start start 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcefc100620 0x7fcefc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcefc100fe0 0x7fcefc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc106630 0x7fcefc19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1119f0 con 0x7fcefc106630 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc111870 con 0x7fcefc100fe0 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf02c7b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc111b70 con 0x7fcefc100620 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf01218640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc106630 0x7fcefc19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf01218640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc106630 0x7fcefc19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59274/0 (socket says 172.21.15.102:59274) 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf01218640 1 -- 172.21.15.102:0/1009284973 learned_addr learned my addr 172.21.15.102:0/1009284973 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcefbfff640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcefc100fe0 0x7fcefc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.695+0000 7fcf00a17640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcefc100620 0x7fcefc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcf01218640 1 -- 172.21.15.102:0/1009284973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcefc100620 msgr2=0x7fcefc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcf01218640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcefc100620 0x7fcefc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcf01218640 1 -- 172.21.15.102:0/1009284973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcefc100fe0 msgr2=0x7fcefc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcf01218640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcefc100fe0 0x7fcefc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcf01218640 1 -- 172.21.15.102:0/1009284973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc19ce70 con 0x7fcefc106630 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcf01218640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc106630 0x7fcefc19c740 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009900 tx=0x7fcee4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcef9ffb640 1 -- 172.21.15.102:0/1009284973 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee403f030 con 0x7fcefc106630 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcef9ffb640 1 -- 172.21.15.102:0/1009284973 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcee40035f0 con 0x7fcefc106630 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcefc19d100 con 0x7fcefc106630 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcef9ffb640 1 -- 172.21.15.102:0/1009284973 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4035070 con 0x7fcefc106630 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.696+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcefc1a49f0 con 0x7fcefc106630 2024-04-24T23:28:16.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.697+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcefc068560 con 0x7fcefc106630 2024-04-24T23:28:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.698+0000 7fcef9ffb640 1 -- 172.21.15.102:0/1009284973 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fcee4003b10 con 0x7fcefc106630 2024-04-24T23:28:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.699+0000 7fcef9ffb640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fcee80a2d80 0x7fcee80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.699+0000 7fcef9ffb640 1 -- 172.21.15.102:0/1009284973 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8623+0+0 (secure 0 0 0) 0x7fcee40eff70 con 0x7fcefc106630 2024-04-24T23:28:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.699+0000 7fcf00a17640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fcee80a2d80 0x7fcee80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:16.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.699+0000 7fcf00a17640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fcee80a2d80 0x7fcee80a5240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcef40058c0 tx=0x7fcef400a2d0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:16.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.703+0000 7fcef9ffb640 1 -- 172.21.15.102:0/1009284973 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcee40bc4e0 con 0x7fcefc106630 2024-04-24T23:28:16.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.914+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcefc108c10 con 0x7fcefc106630 2024-04-24T23:28:16.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.915+0000 7fcef9ffb640 1 -- 172.21.15.102:0/1009284973 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11887 (secure 0 0 0) 0x7fcee40c07e0 con 0x7fcefc106630 2024-04-24T23:28:16.916 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:28:16.916 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1487977595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1487977595},{"type":"v1","addr":"172.21.15.102:6835","nonce":1487977595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1957522131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1957522131},{"type":"v1","addr":"172.21.15.102:6837","nonce":1957522131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":881198064},{"type":"v1","addr":"172.21.15.102:6839","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6841/467071210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6840","nonce":467071210},{"type":"v1","addr":"172.21.15.102:6841","nonce":467071210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2514459470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2514459470},{"type":"v1","addr":"172.21.15.183:6833","nonce":2514459470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/756969355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":756969355},{"type":"v1","addr":"172.21.15.161:6833","nonce":756969355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:11:00.643113+0000","modified":"2024-04-24T23:27:01.805407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24473,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2266462643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2266462643},{"type":"v1","addr":"172.21.15.183:6835","nonce":2266462643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2778927873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2778927873},{"type":"v1","addr":"172.21.15.161:6835","nonce":2778927873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/3933349751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3933349751},{"type":"v1","addr":"172.21.15.183:6837","nonce":3933349751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/395274952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":395274952},{"type":"v1","addr":"172.21.15.183:6839","nonce":395274952}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6837/2386140374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2386140374},{"type":"v1","addr":"172.21.15.161:6837","nonce":2386140374}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/1258581930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1258581930},{"type":"v1","addr":"172.21.15.161:6839","nonce":1258581930}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24473,24497,24472]},"id":1}]} 2024-04-24T23:28:16.917 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 29 2024-04-24T23:28:16.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fcee80a2d80 msgr2=0x7fcee80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fcee80a2d80 0x7fcee80a5240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcef40058c0 tx=0x7fcef400a2d0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc106630 msgr2=0x7fcefc19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:16.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc106630 0x7fcefc19c740 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009900 tx=0x7fcee4004270 comp rx=0 tx=0).stop 2024-04-24T23:28:16.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 shutdown_connections 2024-04-24T23:28:16.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fcee80a2d80 0x7fcee80a5240 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcefc106630 0x7fcefc19c740 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcefc100fe0 0x7fcefc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 --2- 172.21.15.102:0/1009284973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcefc100620 0x7fcefc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:16.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 >> 172.21.15.102:0/1009284973 conn(0x7fcefc077740 msgr2=0x7fcefc102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:16.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 shutdown_connections 2024-04-24T23:28:16.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:16.917+0000 7fcf02c7b640 1 -- 172.21.15.102:0/1009284973 wait complete. 2024-04-24T23:28:16.930 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:28:17.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.027+0000 7f0251116640 1 -- 172.21.15.102:0/65145735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 msgr2=0x7f024c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.027+0000 7f0251116640 1 --2- 172.21.15.102:0/65145735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 0x7f024c100df0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0234009900 tx=0x7f023402a410 comp rx=0 tx=0).stop 2024-04-24T23:28:17.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.027+0000 7f0251116640 1 -- 172.21.15.102:0/65145735 shutdown_connections 2024-04-24T23:28:17.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.027+0000 7f0251116640 1 --2- 172.21.15.102:0/65145735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c101330 0x7f024c10d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.027+0000 7f0251116640 1 --2- 172.21.15.102:0/65145735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 0x7f024c100df0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.027+0000 7f0251116640 1 --2- 172.21.15.102:0/65145735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c106690 0x7f024c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.027+0000 7f0251116640 1 -- 172.21.15.102:0/65145735 >> 172.21.15.102:0/65145735 conn(0x7f024c076440 msgr2=0x7f024c076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.027+0000 7f0251116640 1 -- 172.21.15.102:0/65145735 shutdown_connections 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.027+0000 7f0251116640 1 -- 172.21.15.102:0/65145735 wait complete. 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0251116640 1 Processor -- start 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0251116640 1 -- start start 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0251116640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 0x7f024c196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0251116640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c101330 0x7f024c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0251116640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c106690 0x7f024c19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0251116640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f024c110890 con 0x7f024c100990 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0251116640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f024c110710 con 0x7f024c101330 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0251116640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f024c110a10 con 0x7f024c106690 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 0x7f024c196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 0x7f024c196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59302/0 (socket says 172.21.15.102:59302) 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024ad9d640 1 -- 172.21.15.102:0/3378199253 learned_addr learned my addr 172.21.15.102:0/3378199253 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024b59e640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c106690 0x7f024c19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0243fff640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c101330 0x7f024c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024ad9d640 1 -- 172.21.15.102:0/3378199253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c106690 msgr2=0x7f024c19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024ad9d640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c106690 0x7f024c19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024ad9d640 1 -- 172.21.15.102:0/3378199253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c101330 msgr2=0x7f024c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024ad9d640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c101330 0x7f024c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024ad9d640 1 -- 172.21.15.102:0/3378199253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c19bdb0 con 0x7f024c100990 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f024ad9d640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 0x7f024c196e10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f023c00d770 tx=0x7f023c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.028+0000 7f0248d99640 1 -- 172.21.15.102:0/3378199253 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c017070 con 0x7f024c100990 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.029+0000 7f0248d99640 1 -- 172.21.15.102:0/3378199253 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f023c0054d0 con 0x7f024c100990 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.029+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024c19c0a0 con 0x7f024c100990 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.029+0000 7f0248d99640 1 -- 172.21.15.102:0/3378199253 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c004cf0 con 0x7f024c100990 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.029+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f024c1a3930 con 0x7f024c100990 2024-04-24T23:28:17.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.030+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f024c101d80 con 0x7f024c100990 2024-04-24T23:28:17.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.031+0000 7f0248d99640 1 -- 172.21.15.102:0/3378199253 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f023c004e90 con 0x7f024c100990 2024-04-24T23:28:17.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.031+0000 7f0248d99640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f02380a2d00 0x7f02380a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.032+0000 7f0243fff640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f02380a2d00 0x7f02380a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.032+0000 7f0248d99640 1 -- 172.21.15.102:0/3378199253 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8623+0+0 (secure 0 0 0) 0x7f023c0cee80 con 0x7f024c100990 2024-04-24T23:28:17.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.032+0000 7f0243fff640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f02380a2d00 0x7f02380a51c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0234031440 tx=0x7f02340095f0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:17.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.036+0000 7f0248d99640 1 -- 172.21.15.102:0/3378199253 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f023c09ba40 con 0x7f024c100990 2024-04-24T23:28:17.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.216+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f024c108b00 con 0x7f024c100990 2024-04-24T23:28:17.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.217+0000 7f0248d99640 1 -- 172.21.15.102:0/3378199253 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+23312 (secure 0 0 0) 0x7f023c09b340 con 0x7f024c100990 2024-04-24T23:28:17.218 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-24T23:28:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":108,"fsid":"97eecd66-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:02:53.253279+0000","modified":"2024-04-24T23:28:11.168430+0000","last_up_change":"2024-04-24T23:10:07.423280+0000","last_in_change":"2024-04-24T23:09:46.242425+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:06:16.456403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:10:58.730586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:10:59.855279+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fabc1206-39d7-4c69-9633-e64bf8677c8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6803","nonce":825890094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6805","nonce":825890094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6809","nonce":825890094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":825890094},{"type":"v1","addr":"172.21.15.102:6807","nonce":825890094}]},"public_addr":"172.21.15.102:6803/825890094","cluster_addr":"172.21.15.102:6805/825890094","heartbeat_back_addr":"172.21.15.102:6809/825890094","heartbeat_front_addr":"172.21.15.102:6807/825890094","state":["exists","up"]},{"osd":1,"uuid":"0fbfe050-2056-4fe6-91d6-ab6f6e9702b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6801","nonce":1328568264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6803","nonce":1328568264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6807","nonce":1328568264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1328568264},{"type":"v1","addr":"172.21.15.161:6805","nonce":1328568264}]},"public_addr":"172.21.15.161:6801/1328568264","cluster_addr":"172.21.15.161:6803/1328568264","heartbeat_back_addr":"172.21.15.161:6807/1328568264","heartbeat_front_addr":"172.21.15.161:6805/1328568264","state":["exists","up"]},{"osd":2,"uuid":"673517e2-faa1-4141-9f1c-bb11f16713d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6801","nonce":1173587665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6803","nonce":1173587665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6807","nonce":1173587665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1173587665},{"type":"v1","addr":"172.21.15.183:6805","nonce":1173587665}]},"public_addr":"172.21.15.183:6801/1173587665","cluster_addr":"172.21.15.183:6803/1173587665","heartbeat_back_addr":"172.21.15.183:6807/1173587665","heartbeat_front_addr":"172.21.15.183:6805/1173587665","state":["exists","up"]},{"osd":3,"uuid":"aea12510-9a47-49d7-b0ca-ee38b19c3249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6811","nonce":2459908351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6813","nonce":2459908351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6817","nonce":2459908351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2459908351},{"type":"v1","addr":"172.21.15.102:6815","nonce":2459908351}]},"public_addr":"172.21.15.102:6811/2459908351","cluster_addr":"172.21.15.102:6813/2459908351","heartbeat_back_addr":"172.21.15.102:6817/2459908351","heartbeat_front_addr":"172.21.15.102:6815/2459908351","state":["exists","up"]},{"osd":4,"uuid":"f1d828f2-40e4-4c0b-9ffe-41ceb2115033","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6809","nonce":1666138932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6811","nonce":1666138932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6815","nonce":1666138932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1666138932},{"type":"v1","addr":"172.21.15.161:6813","nonce":1666138932}]},"public_addr":"172.21.15.161:6809/1666138932","cluster_addr":"172.21.15.161:6811/1666138932","heartbeat_back_addr":"172.21.15.161:6815/1666138932","heartbeat_front_addr":"172.21.15.161:6813/1666138932","state":["exists","up"]},{"osd":5,"uuid":"f735dd7c-0d04-4448-846f-53949f04a067","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6809","nonce":351285214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6811","nonce":351285214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6815","nonce":351285214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":351285214},{"type":"v1","addr":"172.21.15.183:6813","nonce":351285214}]},"public_addr":"172.21.15.183:6809/351285214","cluster_addr":"172.21.15.183:6811/351285214","heartbeat_back_addr":"172.21.15.183:6815/351285214","heartbeat_front_addr":"172.21.15.183:6813/351285214","state":["exists","up"]},{"osd":6,"uuid":"74c01493-6457-48d8-807a-0c6108e3d384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6819","nonce":3573356846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6821","nonce":3573356846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6825","nonce":3573356846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3573356846},{"type":"v1","addr":"172.21.15.102:6823","nonce":3573356846}]},"public_addr":"172.21.15.102:6819/3573356846","cluster_addr":"172.21.15.102:6821/3573356846","heartbeat_back_addr":"172.21.15.102:6825/3573356846","heartbeat_front_addr":"172.21.15.102:6823/3573356846","state":["exists","up"]},{"osd":7,"uuid":"761ed995-553c-4b96-9a90-aec367743007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6817","nonce":1449722417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6819","nonce":1449722417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6823","nonce":1449722417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1449722417},{"type":"v1","addr":"172.21.15.161:6821","nonce":1449722417}]},"public_addr":"172.21.15.161:6817/1449722417","cluster_addr":"172.21.15.161:6819/1449722417","heartbeat_back_addr":"172.21.15.161:6823/1449722417","heartbeat_front_addr":"172.21.15.161:6821/1449722417","state":["exists","up"]},{"osd":8,"uuid":"a6396e17-1338-4c85-8e94-523088a41caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6817","nonce":3691469531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6819","nonce":3691469531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6823","nonce":3691469531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3691469531},{"type":"v1","addr":"172.21.15.183:6821","nonce":3691469531}]},"public_addr":"172.21.15.183:6817/3691469531","cluster_addr":"172.21.15.183:6819/3691469531","heartbeat_back_addr":"172.21.15.183:6823/3691469531","heartbeat_front_addr":"172.21.15.183:6821/3691469531","state":["exists","up"]},{"osd":9,"uuid":"e7df24c5-790f-45d6-a588-4de56ecc06d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6827","nonce":3638070029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6829","nonce":3638070029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6833","nonce":3638070029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3638070029},{"type":"v1","addr":"172.21.15.102:6831","nonce":3638070029}]},"public_addr":"172.21.15.102:6827/3638070029","cluster_addr":"172.21.15.102:6829/3638070029","heartbeat_back_addr":"172.21.15.102:6833/3638070029","heartbeat_front_addr":"172.21.15.102:6831/3638070029","state":["exists","up"]},{"osd":10,"uuid":"442f5937-b88f-4000-b961-d097cceb9f88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6825","nonce":968108427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6827","nonce":968108427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6831","nonce":968108427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":968108427},{"type":"v1","addr":"172.21.15.161:6829","nonce":968108427}]},"public_addr":"172.21.15.161:6825/968108427","cluster_addr":"172.21.15.161:6827/968108427","heartbeat_back_addr":"172.21.15.161:6831/968108427","heartbeat_front_addr":"172.21.15.161:6829/968108427","state":["exists","up"]},{"osd":11,"uuid":"46d330b0-0ac0-4bba-b07e-f77fb2a202b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6825","nonce":2953790548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6827","nonce":2953790548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6831","nonce":2953790548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2953790548},{"type":"v1","addr":"172.21.15.183:6829","nonce":2953790548}]},"public_addr":"172.21.15.183:6825/2953790548","cluster_addr":"172.21.15.183:6827/2953790548","heartbeat_back_addr":"172.21.15.183:6831/2953790548","heartbeat_front_addr":"172.21.15.183:6829/2953790548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:24.828365+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:05:48.805152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:12.480100+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:06:38.320094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:03.707445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:28.713278+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:54.644684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:20.185510+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:45.604636+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:12.510914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:38.674704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:04.964667+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/2090298312":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/835175442":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/3958169470":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/3611055373":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6801/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/3702328937":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1394402685":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/4171285045":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/4149172465":"2024-04-25T23:13:01.525573+0000","172.21.15.102:0/1812533171":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6800/3017038970":"2024-04-25T23:03:48.134719+0000","172.21.15.102:0/2445922755":"2024-04-25T23:03:21.040554+0000","172.21.15.102:0/1700818231":"2024-04-25T23:03:48.134719+0000","172.21.15.102:6800/2363268578":"2024-04-25T23:03:21.040554+0000","172.21.15.102:6801/3611055373":"2024-04-25T23:13:01.525573+0000","172.21.15.102:6801/2363268578":"2024-04-25T23:03:21.040554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.219+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f02380a2d00 msgr2=0x7f02380a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.219+0000 7f0251116640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f02380a2d00 0x7f02380a51c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0234031440 tx=0x7f02340095f0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.219+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 msgr2=0x7f024c196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.219+0000 7f0251116640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 0x7f024c196e10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f023c00d770 tx=0x7f023c00dc30 comp rx=0 tx=0).stop 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.219+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 shutdown_connections 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.219+0000 7f0251116640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c106690 0x7f024c19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.220+0000 7f0251116640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c101330 0x7f024c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.220+0000 7f0251116640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f02380a2d00 0x7f02380a51c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.220+0000 7f0251116640 1 --2- 172.21.15.102:0/3378199253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f024c100990 0x7f024c196e10 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.220+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 >> 172.21.15.102:0/3378199253 conn(0x7f024c076440 msgr2=0x7f024c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.220+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 shutdown_connections 2024-04-24T23:28:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.220+0000 7f0251116640 1 -- 172.21.15.102:0/3378199253 wait complete. 2024-04-24T23:28:17.232 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:28:17.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 -- 172.21.15.102:0/1626112795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 msgr2=0x7f106810e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 --2- 172.21.15.102:0/1626112795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 0x7f106810e5e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1060009cd0 tx=0x7f106002a8e0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 -- 172.21.15.102:0/1626112795 shutdown_connections 2024-04-24T23:28:17.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 --2- 172.21.15.102:0/1626112795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 0x7f106810e5e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 --2- 172.21.15.102:0/1626112795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1068101710 0x7f1068101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 --2- 172.21.15.102:0/1626112795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1068107720 0x7f1068107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 -- 172.21.15.102:0/1626112795 >> 172.21.15.102:0/1626112795 conn(0x7f10680fd180 msgr2=0x7f10680ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:17.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 -- 172.21.15.102:0/1626112795 shutdown_connections 2024-04-24T23:28:17.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 -- 172.21.15.102:0/1626112795 wait complete. 2024-04-24T23:28:17.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.330+0000 7f106e5c1640 1 Processor -- start 2024-04-24T23:28:17.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f106e5c1640 1 -- start start 2024-04-24T23:28:17.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f106e5c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1068101710 0x7f1068073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f106e5c1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 0x7f1068073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f106e5c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1068107720 0x7f10680744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f106e5c1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f10681119e0 con 0x7f10681020d0 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f106e5c1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1068111860 con 0x7f1068101710 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f106e5c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1068111b60 con 0x7f1068107720 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f10677fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 0x7f1068073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f10677fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 0x7f1068073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59330/0 (socket says 172.21.15.102:59330) 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f10677fe640 1 -- 172.21.15.102:0/3893315939 learned_addr learned my addr 172.21.15.102:0/3893315939 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f1067fff640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1068101710 0x7f1068073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f106cb5e640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1068107720 0x7f10680744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f10677fe640 1 -- 172.21.15.102:0/3893315939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1068107720 msgr2=0x7f10680744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f10677fe640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1068107720 0x7f10680744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f10677fe640 1 -- 172.21.15.102:0/3893315939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1068101710 msgr2=0x7f1068073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f10677fe640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1068101710 0x7f1068073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f10677fe640 1 -- 172.21.15.102:0/3893315939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10681a6420 con 0x7f10681020d0 2024-04-24T23:28:17.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f1067fff640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1068101710 0x7f1068073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:17.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f106cb5e640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1068107720 0x7f10680744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:17.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.331+0000 7f10677fe640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 0x7f1068073f80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f105800d770 tx=0x7f105800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:17.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.332+0000 7f10657fa640 1 -- 172.21.15.102:0/3893315939 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058017070 con 0x7f10681020d0 2024-04-24T23:28:17.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.332+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10681a6710 con 0x7f10681020d0 2024-04-24T23:28:17.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.332+0000 7f10657fa640 1 -- 172.21.15.102:0/3893315939 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10580054d0 con 0x7f10681020d0 2024-04-24T23:28:17.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.332+0000 7f10657fa640 1 -- 172.21.15.102:0/3893315939 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058004cf0 con 0x7f10681020d0 2024-04-24T23:28:17.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.332+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10681a6c10 con 0x7f10681020d0 2024-04-24T23:28:17.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.332+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10680689b0 con 0x7f10681020d0 2024-04-24T23:28:17.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.334+0000 7f10657fa640 1 -- 172.21.15.102:0/3893315939 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f1058004e90 con 0x7f10681020d0 2024-04-24T23:28:17.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.334+0000 7f10657fa640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f10500a2d00 0x7f10500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.334+0000 7f1067fff640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f10500a2d00 0x7f10500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.334+0000 7f10657fa640 1 -- 172.21.15.102:0/3893315939 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8623+0+0 (secure 0 0 0) 0x7f10580cf420 con 0x7f10681020d0 2024-04-24T23:28:17.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.335+0000 7f1067fff640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f10500a2d00 0x7f10500a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f105c005e00 tx=0x7f105c005c70 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:17.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.338+0000 7f10657fa640 1 -- 172.21.15.102:0/3893315939 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f105809bfe0 con 0x7f10681020d0 2024-04-24T23:28:17.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-mon[33137]: pgmap v477: 97 pgs: 25 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 59 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 819 B/s wr, 7 op/s 2024-04-24T23:28:17.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1436012585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:17.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/1009284973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:17.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.554+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1068109ca0 con 0x7f10681020d0 2024-04-24T23:28:17.556 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:17 smithi102 ceph-mon[29216]: pgmap v477: 97 pgs: 25 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 59 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 819 B/s wr, 7 op/s 2024-04-24T23:28:17.556 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:17 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1436012585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:17.556 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:17 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/1009284973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:17.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.556+0000 7f10657fa640 1 -- 172.21.15.102:0/3893315939 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f105809b8e0 con 0x7f10681020d0 2024-04-24T23:28:17.557 INFO:teuthology.orchestra.run.smithi102.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:28:17.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f10500a2d00 msgr2=0x7f10500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f10500a2d00 0x7f10500a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f105c005e00 tx=0x7f105c005c70 comp rx=0 tx=0).stop 2024-04-24T23:28:17.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 msgr2=0x7f1068073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 0x7f1068073f80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f105800d770 tx=0x7f105800dc30 comp rx=0 tx=0).stop 2024-04-24T23:28:17.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 shutdown_connections 2024-04-24T23:28:17.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1068107720 0x7f10680744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f10500a2d00 0x7f10500a51c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10681020d0 0x7f1068073f80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 --2- 172.21.15.102:0/3893315939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1068101710 0x7f1068073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 >> 172.21.15.102:0/3893315939 conn(0x7f10680fd180 msgr2=0x7f10680fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:17.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 shutdown_connections 2024-04-24T23:28:17.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.559+0000 7f106e5c1640 1 -- 172.21.15.102:0/3893315939 wait complete. 2024-04-24T23:28:17.571 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:28:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-mon[32709]: pgmap v477: 97 pgs: 25 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 59 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 819 B/s wr, 7 op/s 2024-04-24T23:28:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1436012585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:17.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/1009284973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:28:17.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.671+0000 7f704569e640 1 -- 172.21.15.102:0/549126993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7040100db0 msgr2=0x7f7040101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.671+0000 7f704569e640 1 --2- 172.21.15.102:0/549126993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7040100db0 0x7f7040101230 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7034009900 tx=0x7f703402a410 comp rx=0 tx=0).stop 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.671+0000 7f704569e640 1 -- 172.21.15.102:0/549126993 shutdown_connections 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.671+0000 7f704569e640 1 --2- 172.21.15.102:0/549126993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7040101770 0x7f704010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.671+0000 7f704569e640 1 --2- 172.21.15.102:0/549126993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7040100db0 0x7f7040101230 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.671+0000 7f704569e640 1 --2- 172.21.15.102:0/549126993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70401055f0 0x7f70401059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.671+0000 7f704569e640 1 -- 172.21.15.102:0/549126993 >> 172.21.15.102:0/549126993 conn(0x7f7040076bd0 msgr2=0x7f70400fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.671+0000 7f704569e640 1 -- 172.21.15.102:0/549126993 shutdown_connections 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.671+0000 7f704569e640 1 -- 172.21.15.102:0/549126993 wait complete. 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f704569e640 1 Processor -- start 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f704569e640 1 -- start start 2024-04-24T23:28:17.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f704569e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7040100db0 0x7f70400fc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f704569e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7040101770 0x7f70400fc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f704569e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70401055f0 0x7f70400fca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f704569e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f70401108e0 con 0x7f70401055f0 2024-04-24T23:28:17.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f704569e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7040110760 con 0x7f7040100db0 2024-04-24T23:28:17.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f704569e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7040110a60 con 0x7f7040101770 2024-04-24T23:28:17.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f703f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70401055f0 0x7f70400fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f703f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70401055f0 0x7f70400fca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59344/0 (socket says 172.21.15.102:59344) 2024-04-24T23:28:17.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f703f7fe640 1 -- 172.21.15.102:0/4210236422 learned_addr learned my addr 172.21.15.102:0/4210236422 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:17.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f703effd640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7040100db0 0x7f70400fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.672+0000 7f703e7fc640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7040101770 0x7f70400fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f703f7fe640 1 -- 172.21.15.102:0/4210236422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7040101770 msgr2=0x7f70400fc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f703f7fe640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7040101770 0x7f70400fc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f703f7fe640 1 -- 172.21.15.102:0/4210236422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7040100db0 msgr2=0x7f70400fc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f703f7fe640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7040100db0 0x7f70400fc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f703f7fe640 1 -- 172.21.15.102:0/4210236422 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70401a6450 con 0x7f70401055f0 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f703effd640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7040100db0 0x7f70400fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f703f7fe640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70401055f0 0x7f70400fca90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f703800cad0 tx=0x7f703800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f7027fff640 1 -- 172.21.15.102:0/4210236422 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703800ae20 con 0x7f70401055f0 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70401a6740 con 0x7f70401055f0 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f7027fff640 1 -- 172.21.15.102:0/4210236422 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7038005020 con 0x7f70401055f0 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f7027fff640 1 -- 172.21.15.102:0/4210236422 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703800f6b0 con 0x7f70401055f0 2024-04-24T23:28:17.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.673+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70401a6c10 con 0x7f70401055f0 2024-04-24T23:28:17.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.674+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70400689b0 con 0x7f70401055f0 2024-04-24T23:28:17.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.675+0000 7f7027fff640 1 -- 172.21.15.102:0/4210236422 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f70380051d0 con 0x7f70401055f0 2024-04-24T23:28:17.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.675+0000 7f7027fff640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f70280a2d00 0x7f70280a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:17.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.676+0000 7f703effd640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f70280a2d00 0x7f70280a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:17.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.676+0000 7f7027fff640 1 -- 172.21.15.102:0/4210236422 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8837+0+0 (secure 0 0 0) 0x7f70380cf630 con 0x7f70401055f0 2024-04-24T23:28:17.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.676+0000 7f703effd640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f70280a2d00 0x7f70280a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7030009930 tx=0x7f7030008040 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:17.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.680+0000 7f7027fff640 1 -- 172.21.15.102:0/4210236422 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7038014030 con 0x7f70401055f0 2024-04-24T23:28:17.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:17 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a[29211]: 2024-04-24T23:28:17.554+0000 7f7ac9ec9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:28:17.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:17.902+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f704002d170 con 0x7f70401055f0 2024-04-24T23:28:18.011 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f[63071]: ignoring --setuser ceph since I am not root 2024-04-24T23:28:18.011 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f[63071]: ignoring --setgroup ceph since I am not root 2024-04-24T23:28:18.012 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-mds[63075]: -- 172.21.15.183:0/3427618862 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555d4e83a4e0 con 0x555d4e811000 2024-04-24T23:28:18.012 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-mds[63075]: -- 172.21.15.183:0/3427618862 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d4e83da40 con 0x555d4e811000 2024-04-24T23:28:18.012 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f[63071]: starting mds.f at 2024-04-24T23:28:18.012 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i[63686]: ignoring --setuser ceph since I am not root 2024-04-24T23:28:18.012 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i[63686]: ignoring --setgroup ceph since I am not root 2024-04-24T23:28:18.012 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-mds[63690]: -- 172.21.15.183:0/1419149926 <== mon.0 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562020eb3a40 con 0x562020e86c00 2024-04-24T23:28:18.012 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i[63686]: starting mds.i at 2024-04-24T23:28:18.013 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c[63377]: ignoring --setuser ceph since I am not root 2024-04-24T23:28:18.013 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c[63377]: ignoring --setgroup ceph since I am not root 2024-04-24T23:28:18.013 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-mds[63381]: -- 172.21.15.183:0/1434535869 <== mon.0 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5628b099ba40 con 0x5628b096f000 2024-04-24T23:28:18.013 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:28:17 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c[63377]: starting mds.c at 2024-04-24T23:28:18.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:18 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3378199253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:18.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:18 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3893315939' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:28:18.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:18 smithi161 ceph-mon[32709]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:28:18.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:18 smithi161 ceph-mon[32709]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:28:18.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:18 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/3893315939' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:28:18.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:18 smithi161 ceph-mon[32709]: osdmap e109: 12 total, 12 up, 12 in 2024-04-24T23:28:18.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:18 smithi161 ceph-mon[32709]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T23:28:18.359 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:18 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4210236422' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:28:18.359 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h[62698]: ignoring --setuser ceph since I am not root 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h[62698]: ignoring --setgroup ceph since I am not root 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-mds[62702]: -- 172.21.15.161:0/2660826903 <== mon.0 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc1c259a40 con 0x55bc1c22c800 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h[62698]: starting mds.h at 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e[62390]: ignoring --setuser ceph since I am not root 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e[62390]: ignoring --setgroup ceph since I am not root 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-mds[62394]: -- 172.21.15.161:0/572662280 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d16d75a40 con 0x564d16d48800 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e[62390]: starting mds.e at 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b[63009]: ignoring --setuser ceph since I am not root 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b[63009]: ignoring --setgroup ceph since I am not root 2024-04-24T23:28:18.360 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-mds[63013]: -- 172.21.15.161:0/2865123850 <== mon.0 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c470ceba40 con 0x55c470cbe800 2024-04-24T23:28:18.361 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:28:17 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b[63009]: starting mds.b at 2024-04-24T23:28:18.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:18 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3378199253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:18.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:18 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3893315939' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:28:18.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:18 smithi183 ceph-mon[33137]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:28:18.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:18 smithi183 ceph-mon[33137]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:28:18.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:18 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/3893315939' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:28:18.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:18 smithi183 ceph-mon[33137]: osdmap e109: 12 total, 12 up, 12 in 2024-04-24T23:28:18.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:18 smithi183 ceph-mon[33137]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T23:28:18.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:18 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4210236422' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:28:18.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.562+0000 7f7027fff640 1 -- 172.21.15.102:0/4210236422 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f703809c110 con 0x7f70401055f0 2024-04-24T23:28:18.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f70280a2d00 msgr2=0x7f70280a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f70280a2d00 0x7f70280a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7030009930 tx=0x7f7030008040 comp rx=0 tx=0).stop 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70401055f0 msgr2=0x7f70400fca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70401055f0 0x7f70400fca90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f703800cad0 tx=0x7f703800cf90 comp rx=0 tx=0).stop 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 shutdown_connections 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f70280a2d00 0x7f70280a51c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70401055f0 0x7f70400fca90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7040101770 0x7f70400fc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 --2- 172.21.15.102:0/4210236422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7040100db0 0x7f70400fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.565+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 >> 172.21.15.102:0/4210236422 conn(0x7f7040076bd0 msgr2=0x7f7040100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.566+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 shutdown_connections 2024-04-24T23:28:18.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.566+0000 7f704569e640 1 -- 172.21.15.102:0/4210236422 wait complete. 2024-04-24T23:28:18.570 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:18 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3378199253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:28:18.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:18 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3893315939' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:28:18.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:18 smithi102 ceph-mon[29216]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:28:18.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:18 smithi102 ceph-mon[29216]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:28:18.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:18 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/3893315939' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:28:18.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:18 smithi102 ceph-mon[29216]: osdmap e109: 12 total, 12 up, 12 in 2024-04-24T23:28:18.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:18 smithi102 ceph-mon[29216]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T23:28:18.571 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:18 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4210236422' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:28:18.579 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:28:18.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.682+0000 7f3b7dc76640 1 -- 172.21.15.102:0/2459396844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 msgr2=0x7f3b7810e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.682+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/2459396844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 0x7f3b7810e5e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009ca0 tx=0x7f3b7002a950 comp rx=0 tx=0).stop 2024-04-24T23:28:18.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.682+0000 7f3b7dc76640 1 -- 172.21.15.102:0/2459396844 shutdown_connections 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.682+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/2459396844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 0x7f3b7810e5e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.682+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/2459396844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b78069d40 0x7f3b7806a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.682+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/2459396844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b78107940 0x7f3b78107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.682+0000 7f3b7dc76640 1 -- 172.21.15.102:0/2459396844 >> 172.21.15.102:0/2459396844 conn(0x7f3b78076440 msgr2=0x7f3b78076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.682+0000 7f3b7dc76640 1 -- 172.21.15.102:0/2459396844 shutdown_connections 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.682+0000 7f3b7dc76640 1 -- 172.21.15.102:0/2459396844 wait complete. 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b7dc76640 1 Processor -- start 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b7dc76640 1 -- start start 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b7dc76640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b78069d40 0x7f3b7819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b7dc76640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 0x7f3b7819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b7dc76640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b78107940 0x7f3b78197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b7dc76640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781119a0 con 0x7f3b7806a700 2024-04-24T23:28:18.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b7dc76640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78111820 con 0x7f3b78107940 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b7dc76640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78111b20 con 0x7f3b78069d40 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b76ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 0x7f3b7819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b76ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 0x7f3b7819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59372/0 (socket says 172.21.15.102:59372) 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b76ffd640 1 -- 172.21.15.102:0/295479493 learned_addr learned my addr 172.21.15.102:0/295479493 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b777fe640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b78069d40 0x7f3b7819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.683+0000 7f3b77fff640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b78107940 0x7f3b78197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b76ffd640 1 -- 172.21.15.102:0/295479493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b78069d40 msgr2=0x7f3b7819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b76ffd640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b78069d40 0x7f3b7819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b76ffd640 1 -- 172.21.15.102:0/295479493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b78107940 msgr2=0x7f3b78197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b76ffd640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b78107940 0x7f3b78197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b76ffd640 1 -- 172.21.15.102:0/295479493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b78198420 con 0x7f3b7806a700 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b76ffd640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 0x7f3b7819df90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00c920 tx=0x7f3b6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b74ff9640 1 -- 172.21.15.102:0/295479493 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c007c70 con 0x7f3b7806a700 2024-04-24T23:28:18.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b78198710 con 0x7f3b7806a700 2024-04-24T23:28:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b74ff9640 1 -- 172.21.15.102:0/295479493 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b6c007e10 con 0x7f3b7806a700 2024-04-24T23:28:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b74ff9640 1 -- 172.21.15.102:0/295479493 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c018820 con 0x7f3b7806a700 2024-04-24T23:28:18.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.684+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781a49d0 con 0x7f3b7806a700 2024-04-24T23:28:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.685+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b78069590 con 0x7f3b7806a700 2024-04-24T23:28:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.686+0000 7f3b74ff9640 1 -- 172.21.15.102:0/295479493 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7f3b6c0189c0 con 0x7f3b7806a700 2024-04-24T23:28:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.686+0000 7f3b74ff9640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3b600a2d00 0x7f3b600a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.687+0000 7f3b777fe640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3b600a2d00 0x7f3b600a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.687+0000 7f3b74ff9640 1 -- 172.21.15.102:0/295479493 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8837+0+0 (secure 0 0 0) 0x7f3b6c0cf790 con 0x7f3b7806a700 2024-04-24T23:28:18.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.687+0000 7f3b777fe640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3b600a2d00 0x7f3b600a51c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b68005e10 tx=0x7f3b68005da0 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:18.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.691+0000 7f3b74ff9640 1 -- 172.21.15.102:0/295479493 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3b6c014030 con 0x7f3b7806a700 2024-04-24T23:28:18.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:18.868+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3b781a4ba0 con 0x7f3b7806a700 2024-04-24T23:28:19.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: pgmap v479: 97 pgs: 18 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 70 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 895 B/s wr, 6 op/s 2024-04-24T23:28:19.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: Cluster is now healthy 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4210236422' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.183:6834/2114984348,v1:172.21.15.183:6837/2114984348] up:boot 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.183:6835/4188457848,v1:172.21.15.183:6839/4188457848] up:boot 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.183:6836/679608065,v1:172.21.15.183:6838/679608065] up:boot 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.161:6834/1017572720,v1:172.21.15.161:6837/1017572720] up:boot 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.161:6836/1643841070,v1:172.21.15.161:6838/1643841070] up:boot 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: mds.? [v2:172.21.15.161:6835/1697530429,v1:172.21.15.161:6839/1697530429] up:boot 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: fsmap 12 up:standby 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:28:19.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:28:19.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:28:19.513 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:19 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/295479493' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:28:19.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.566+0000 7f3b74ff9640 1 -- 172.21.15.102:0/295479493 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) ==== 181+0+0 (secure 0 0 0) 0x7f3b6c00de20 con 0x7f3b7806a700 2024-04-24T23:28:19.567 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:28:19.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3b600a2d00 msgr2=0x7f3b600a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3b600a2d00 0x7f3b600a51c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b68005e10 tx=0x7f3b68005da0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 msgr2=0x7f3b7819df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 0x7f3b7819df90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00c920 tx=0x7f3b6c00cde0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 shutdown_connections 2024-04-24T23:28:19.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b78107940 0x7f3b78197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7f3b600a2d00 0x7f3b600a51c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b7806a700 0x7f3b7819df90 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 --2- 172.21.15.102:0/295479493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b78069d40 0x7f3b7819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 >> 172.21.15.102:0/295479493 conn(0x7f3b78076440 msgr2=0x7f3b780ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:19.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 shutdown_connections 2024-04-24T23:28:19.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.569+0000 7f3b7dc76640 1 -- 172.21.15.102:0/295479493 wait complete. 2024-04-24T23:28:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: pgmap v479: 97 pgs: 18 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 70 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 895 B/s wr, 6 op/s 2024-04-24T23:28:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:28:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:28:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: Cluster is now healthy 2024-04-24T23:28:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4210236422' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:28:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.183:6834/2114984348,v1:172.21.15.183:6837/2114984348] up:boot 2024-04-24T23:28:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.183:6835/4188457848,v1:172.21.15.183:6839/4188457848] up:boot 2024-04-24T23:28:19.572 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.183:6836/679608065,v1:172.21.15.183:6838/679608065] up:boot 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.161:6834/1017572720,v1:172.21.15.161:6837/1017572720] up:boot 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.161:6836/1643841070,v1:172.21.15.161:6838/1643841070] up:boot 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: mds.? [v2:172.21.15.161:6835/1697530429,v1:172.21.15.161:6839/1697530429] up:boot 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: fsmap 12 up:standby 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:28:19.573 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:19 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/295479493' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:28:19.591 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: pgmap v479: 97 pgs: 18 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 70 active+clean; 17 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 895 B/s wr, 6 op/s 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: Cluster is now healthy 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4210236422' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.183:6834/2114984348,v1:172.21.15.183:6837/2114984348] up:boot 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.183:6835/4188457848,v1:172.21.15.183:6839/4188457848] up:boot 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.183:6836/679608065,v1:172.21.15.183:6838/679608065] up:boot 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.161:6834/1017572720,v1:172.21.15.161:6837/1017572720] up:boot 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.161:6836/1643841070,v1:172.21.15.161:6838/1643841070] up:boot 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: mds.? [v2:172.21.15.161:6835/1697530429,v1:172.21.15.161:6839/1697530429] up:boot 2024-04-24T23:28:19.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: fsmap 12 up:standby 2024-04-24T23:28:19.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:28:19.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:28:19.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:28:19.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:28:19.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:28:19.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: from='mgr.15288 172.21.15.102:0/3928338026' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:28:19.610 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:19 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/295479493' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.697+0000 7fdf2139e640 1 -- 172.21.15.102:0/1539210091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c100610 msgr2=0x7fdf1c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.697+0000 7fdf2139e640 1 --2- 172.21.15.102:0/1539210091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c100610 0x7fdf1c100a90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009930 tx=0x7fdf1002a420 comp rx=0 tx=0).stop 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.697+0000 7fdf2139e640 1 -- 172.21.15.102:0/1539210091 shutdown_connections 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.697+0000 7fdf2139e640 1 --2- 172.21.15.102:0/1539210091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1c100fd0 0x7fdf1c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.697+0000 7fdf2139e640 1 --2- 172.21.15.102:0/1539210091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c100610 0x7fdf1c100a90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.697+0000 7fdf2139e640 1 --2- 172.21.15.102:0/1539210091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1c1066b0 0x7fdf1c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.697+0000 7fdf2139e640 1 -- 172.21.15.102:0/1539210091 >> 172.21.15.102:0/1539210091 conn(0x7fdf1c077740 msgr2=0x7fdf1c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.697+0000 7fdf2139e640 1 -- 172.21.15.102:0/1539210091 shutdown_connections 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.697+0000 7fdf2139e640 1 -- 172.21.15.102:0/1539210091 wait complete. 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.698+0000 7fdf2139e640 1 Processor -- start 2024-04-24T23:28:19.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.698+0000 7fdf2139e640 1 -- start start 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.698+0000 7fdf2139e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1c100610 0x7fdf1c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.698+0000 7fdf2139e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1c100fd0 0x7fdf1c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.698+0000 7fdf2139e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c1066b0 0x7fdf1c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.698+0000 7fdf2139e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1108e0 con 0x7fdf1c1066b0 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.698+0000 7fdf2139e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c110760 con 0x7fdf1c100fd0 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.698+0000 7fdf2139e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c110a60 con 0x7fdf1c100610 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.698+0000 7fdf1b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c1066b0 0x7fdf1c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c1066b0 0x7fdf1c19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59394/0 (socket says 172.21.15.102:59394) 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1b7fe640 1 -- 172.21.15.102:0/4109143642 learned_addr learned my addr 172.21.15.102:0/4109143642 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1affd640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1c100610 0x7fdf1c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1a7fc640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1c100fd0 0x7fdf1c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1b7fe640 1 -- 172.21.15.102:0/4109143642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1c100610 msgr2=0x7fdf1c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1b7fe640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1c100610 0x7fdf1c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1b7fe640 1 -- 172.21.15.102:0/4109143642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1c100fd0 msgr2=0x7fdf1c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1b7fe640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1c100fd0 0x7fdf1c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1b7fe640 1 -- 172.21.15.102:0/4109143642 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c19ced0 con 0x7fdf1c1066b0 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1a7fc640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1c100fd0 0x7fdf1c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf1b7fe640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c1066b0 0x7fdf1c19c7a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d750 tx=0x7fdf1400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf03fff640 1 -- 172.21.15.102:0/4109143642 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14017070 con 0x7fdf1c1066b0 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c19d1c0 con 0x7fdf1c1066b0 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf03fff640 1 -- 172.21.15.102:0/4109143642 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf140054d0 con 0x7fdf1c1066b0 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf03fff640 1 -- 172.21.15.102:0/4109143642 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14004c60 con 0x7fdf1c1066b0 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.699+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c1a4a50 con 0x7fdf1c1066b0 2024-04-24T23:28:19.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.700+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c068560 con 0x7fdf1c1066b0 2024-04-24T23:28:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.701+0000 7fdf03fff640 1 -- 172.21.15.102:0/4109143642 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 137965+0+0 (secure 0 0 0) 0x7fdf14004e00 con 0x7fdf1c1066b0 2024-04-24T23:28:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.702+0000 7fdf03fff640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdf040a2d00 0x7fdf040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:28:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.702+0000 7fdf1affd640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdf040a2d00 0x7fdf040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:28:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.702+0000 7fdf03fff640 1 -- 172.21.15.102:0/4109143642 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8283+0+0 (secure 0 0 0) 0x7fdf140cedd0 con 0x7fdf1c1066b0 2024-04-24T23:28:19.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.702+0000 7fdf1affd640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdf040a2d00 0x7fdf040a51c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c005c40 tx=0x7fdf0c00a780 comp rx=0 tx=0).ready entity=mgr.15288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:28:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.707+0000 7fdf03fff640 1 -- 172.21.15.102:0/4109143642 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdf1409bae0 con 0x7fdf1c1066b0 2024-04-24T23:28:19.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:19.885+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf1c1a4d30 con 0x7fdf1c1066b0 2024-04-24T23:28:20.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.572+0000 7fdf03fff640 1 -- 172.21.15.102:0/4109143642 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) ==== 169+0+0 (secure 0 0 0) 0x7fdf1409b3e0 con 0x7fdf1c1066b0 2024-04-24T23:28:20.573 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' removed 2024-04-24T23:28:20.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.575+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdf040a2d00 msgr2=0x7fdf040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.575+0000 7fdf2139e640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdf040a2d00 0x7fdf040a51c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c005c40 tx=0x7fdf0c00a780 comp rx=0 tx=0).stop 2024-04-24T23:28:20.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.575+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c1066b0 msgr2=0x7fdf1c19c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:28:20.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.575+0000 7fdf2139e640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c1066b0 0x7fdf1c19c7a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d750 tx=0x7fdf1400dc10 comp rx=0 tx=0).stop 2024-04-24T23:28:20.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.576+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 shutdown_connections 2024-04-24T23:28:20.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.576+0000 7fdf2139e640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:6800/161836032,v1:172.21.15.102:6801/161836032] conn(0x7fdf040a2d00 0x7fdf040a51c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.576+0000 7fdf2139e640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf1c1066b0 0x7fdf1c19c7a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.576+0000 7fdf2139e640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1c100fd0 0x7fdf1c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.576+0000 7fdf2139e640 1 --2- 172.21.15.102:0/4109143642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf1c100610 0x7fdf1c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:28:20.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.576+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 >> 172.21.15.102:0/4109143642 conn(0x7fdf1c077740 msgr2=0x7fdf1c0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:28:20.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.576+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 shutdown_connections 2024-04-24T23:28:20.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-24T23:28:20.577+0000 7fdf2139e640 1 -- 172.21.15.102:0/4109143642 wait complete. 2024-04-24T23:28:20.596 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:28:20.624 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:28:20.663 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:28:20.694 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T23:28:20.694 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:28:20.717 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:28:20.736 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:28:20.753 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T23:28:20.753 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-24T23:28:20.753 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.a 2024-04-24T23:28:20.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:20 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/295479493' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:28:20.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:20 smithi161 ceph-mon[32709]: osdmap e110: 12 total, 12 up, 12 in 2024-04-24T23:28:20.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:20 smithi161 ceph-mon[32709]: from='client.? 172.21.15.102:0/4109143642' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:28:21.011 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:20 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/295479493' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:28:21.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:20 smithi183 ceph-mon[33137]: osdmap e110: 12 total, 12 up, 12 in 2024-04-24T23:28:21.012 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:20 smithi183 ceph-mon[33137]: from='client.? 172.21.15.102:0/4109143642' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:28:21.071 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:20 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/295479493' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:28:21.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:20 smithi102 ceph-mon[29216]: osdmap e110: 12 total, 12 up, 12 in 2024-04-24T23:28:21.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:20 smithi102 ceph-mon[29216]: from='client.? 172.21.15.102:0/4109143642' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:28:21.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:20 smithi102 systemd[1]: Stopping Ceph mon.a for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:28:21.554 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:21 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a[29211]: 2024-04-24T23:28:21.185+0000 7f7acf6d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:28:21.554 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:21 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a[29211]: 2024-04-24T23:28:21.185+0000 7f7acf6d4700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T23:28:21.554 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:21 smithi102 podman[131795]: 2024-04-24 23:28:21.228571739 +0000 UTC m=+0.162727856 container died 4c8285e9b32c4e37c23d8aa1ecff5baa73383cd6f451a57b77de1fe878ecd948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9) 2024-04-24T23:28:21.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:21 smithi102 podman[131795]: 2024-04-24 23:28:21.660510508 +0000 UTC m=+0.594666621 container cleanup 4c8285e9b32c4e37c23d8aa1ecff5baa73383cd6f451a57b77de1fe878ecd948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:28:21.821 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:21 smithi102 bash[131795]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a 2024-04-24T23:28:22.321 INFO:journalctl@ceph.mon.a.smithi102.stdout:Apr 24 23:28:21 smithi102 podman[131809]: 2024-04-24 23:28:21.991907687 +0000 UTC m=+0.763447143 container remove 4c8285e9b32c4e37c23d8aa1ecff5baa73383cd6f451a57b77de1fe878ecd948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T23:28:22.453 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.a.service' 2024-04-24T23:28:22.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:28:22.644 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-24T23:28:22.644 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-24T23:28:22.644 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.b 2024-04-24T23:28:23.108 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:22 smithi161 systemd[1]: Stopping Ceph mon.b for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:28:25.312 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:25 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-b[32705]: 2024-04-24T23:28:25.040+0000 7f83160cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:28:25.312 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:25 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-b[32705]: 2024-04-24T23:28:25.040+0000 7f83160cd700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-24T23:28:25.608 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:25 smithi161 podman[68961]: 2024-04-24 23:28:25.312382063 +0000 UTC m=+1.869291739 container died b2b0f3439b1c697cea178047c238b90273801a0170bd01ade829e03cce99df9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:28:30.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:30 smithi161 podman[68961]: 2024-04-24 23:28:30.270067239 +0000 UTC m=+6.826976917 container cleanup b2b0f3439b1c697cea178047c238b90273801a0170bd01ade829e03cce99df9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:28:30.609 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:30 smithi161 bash[68961]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-b 2024-04-24T23:28:33.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 24 23:28:33 smithi161 podman[68971]: 2024-04-24 23:28:33.464387259 +0000 UTC m=+8.146692696 container remove b2b0f3439b1c697cea178047c238b90273801a0170bd01ade829e03cce99df9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-24T23:28:34.621 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.b.service' 2024-04-24T23:28:34.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:28:34.793 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-24T23:28:34.793 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-24T23:28:34.793 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.c 2024-04-24T23:28:35.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:34 smithi183 systemd[1]: Stopping Ceph mon.c for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:28:38.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:38 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-c[33119]: 2024-04-24T23:28:38.091+0000 7f1ff918c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:28:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:38 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-c[33119]: 2024-04-24T23:28:38.091+0000 7f1ff918c700 -1 mon.c@1(probing) e3 *** Got Signal Terminated *** 2024-04-24T23:28:38.512 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:38 smithi183 podman[70495]: 2024-04-24 23:28:38.286457048 +0000 UTC m=+2.459241523 container died daac70bc659d7763018fc46f08325a8e3659d69a77a8d21bb1922634206a966e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T23:28:44.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:44 smithi183 podman[70495]: 2024-04-24 23:28:44.375491869 +0000 UTC m=+8.548276337 container cleanup daac70bc659d7763018fc46f08325a8e3659d69a77a8d21bb1922634206a966e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:28:44.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 24 23:28:44 smithi183 bash[70495]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mon-c 2024-04-24T23:28:47.022 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mon.c.service' 2024-04-24T23:28:47.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:28:47.091 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-24T23:28:47.091 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-24T23:28:47.091 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.x 2024-04-24T23:28:47.536 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:28:47 smithi102 systemd[1]: Stopping Ceph mgr.x for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:28:49.821 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:28:49 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x[29459]: 2024-04-24T23:28:49.350+0000 7f8bed181700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:28:49.821 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:28:49 smithi102 podman[131917]: 2024-04-24 23:28:49.392599404 +0000 UTC m=+1.979542710 container died 78ac8b135b5e5acb061a03d8b5063b011ee1d01c565d7c90e6c3b7dc010b491d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-24T23:28:54.571 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:28:54 smithi102 podman[131917]: 2024-04-24 23:28:54.123404911 +0000 UTC m=+6.710348214 container cleanup 78ac8b135b5e5acb061a03d8b5063b011ee1d01c565d7c90e6c3b7dc010b491d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T23:28:54.571 INFO:journalctl@ceph.mgr.x.smithi102.stdout:Apr 24 23:28:54 smithi102 bash[131917]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-x 2024-04-24T23:28:55.179 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-24T23:28:55.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:28:55.215 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-24T23:28:55.215 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-24T23:28:55.215 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.y 2024-04-24T23:28:55.608 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:28:55 smithi161 systemd[1]: Stopping Ceph mgr.y for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:28:57.859 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:28:57 smithi161 podman[69087]: 2024-04-24 23:28:57.572055352 +0000 UTC m=+1.901514430 container died 0d561e3591389cbb20303ffb1fc5866f12bec25bffbd86b0f27e8f5e6f768624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T23:29:03.358 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:29:03 smithi161 podman[69087]: 2024-04-24 23:29:03.063614015 +0000 UTC m=+7.393073091 container cleanup 0d561e3591389cbb20303ffb1fc5866f12bec25bffbd86b0f27e8f5e6f768624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:29:03.359 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 24 23:29:03 smithi161 bash[69087]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-y 2024-04-24T23:29:05.953 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-24T23:29:05.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:29:05.989 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-24T23:29:05.989 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-24T23:29:05.989 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.z 2024-04-24T23:29:06.476 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:29:06 smithi183 systemd[1]: Stopping Ceph mgr.z for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:29:07.754 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:29:07 smithi183 podman[70630]: 2024-04-24 23:29:07.418146913 +0000 UTC m=+1.228574215 container died a929bbfac236eff30661a490426103df687e9f21d8e0650b905c89ea072f7a57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:29:13.511 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:29:13 smithi183 podman[70630]: 2024-04-24 23:29:13.042696043 +0000 UTC m=+6.853123351 container cleanup a929bbfac236eff30661a490426103df687e9f21d8e0650b905c89ea072f7a57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-24T23:29:13.511 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 24 23:29:13 smithi183 bash[70630]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mgr-z 2024-04-24T23:29:16.499 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-24T23:29:16.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:29:16.534 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-24T23:29:16.534 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T23:29:16.534 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.0 2024-04-24T23:29:16.821 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:16 smithi102 systemd[1]: Stopping Ceph osd.0 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:29:17.321 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:16 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0[44288]: 2024-04-24T23:29:16.946+0000 7f31e5493700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:29:17.321 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:16 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0[44288]: 2024-04-24T23:29:16.946+0000 7f31e5493700 -1 osd.0 111 *** Got signal Terminated *** 2024-04-24T23:29:17.321 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:16 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0[44288]: 2024-04-24T23:29:16.946+0000 7f31e5493700 -1 osd.0 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:29:22.784 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:22 smithi102 podman[132053]: 2024-04-24 23:29:22.463931489 +0000 UTC m=+5.740036980 container died 45a701edf386c9fe523efec886559195fdc56cbb6dea07a41910ea2170610da8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T23:29:23.071 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:22 smithi102 podman[132053]: 2024-04-24 23:29:22.879388833 +0000 UTC m=+6.155494328 container cleanup 45a701edf386c9fe523efec886559195fdc56cbb6dea07a41910ea2170610da8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T23:29:23.071 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:22 smithi102 bash[132053]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0 2024-04-24T23:29:23.454 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:23 smithi102 podman[132082]: 2024-04-24 23:29:23.139429527 +0000 UTC m=+0.659343586 container remove 45a701edf386c9fe523efec886559195fdc56cbb6dea07a41910ea2170610da8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:29:23.751 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:23 smithi102 podman[132172]: 2024-04-24 23:29:23.357318478 +0000 UTC m=+0.018147702 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:29:23.752 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:23 smithi102 podman[132172]: 2024-04-24 23:29:23.498824641 +0000 UTC m=+0.159653901 container create a2f5d74afe411ebd4efdd2528c351149b375a9c39c5cda70152f3133dbfd905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-24T23:29:24.071 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:23 smithi102 podman[132172]: 2024-04-24 23:29:23.7518655 +0000 UTC m=+0.412694718 container init a2f5d74afe411ebd4efdd2528c351149b375a9c39c5cda70152f3133dbfd905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T23:29:24.071 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:23 smithi102 podman[132172]: 2024-04-24 23:29:23.755162114 +0000 UTC m=+0.415991329 container start a2f5d74afe411ebd4efdd2528c351149b375a9c39c5cda70152f3133dbfd905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-24T23:29:24.071 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:23 smithi102 podman[132172]: 2024-04-24 23:29:23.910510842 +0000 UTC m=+0.571340073 container attach a2f5d74afe411ebd4efdd2528c351149b375a9c39c5cda70152f3133dbfd905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:29:24.571 INFO:journalctl@ceph.osd.0.smithi102.stdout:Apr 24 23:29:24 smithi102 podman[132172]: 2024-04-24 23:29:24.317555104 +0000 UTC m=+0.978384320 container died a2f5d74afe411ebd4efdd2528c351149b375a9c39c5cda70152f3133dbfd905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T23:29:24.974 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T23:29:25.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:29:25.015 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T23:29:25.015 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T23:29:25.015 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.1 2024-04-24T23:29:25.336 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:25 smithi161 systemd[1]: Stopping Ceph osd.1 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:29:25.608 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:25 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1[37270]: 2024-04-24T23:29:25.335+0000 7fbd8b5ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:29:25.608 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:25 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1[37270]: 2024-04-24T23:29:25.335+0000 7fbd8b5ab700 -1 osd.1 111 *** Got signal Terminated *** 2024-04-24T23:29:25.608 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:25 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1[37270]: 2024-04-24T23:29:25.335+0000 7fbd8b5ab700 -1 osd.1 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:29:31.222 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:30 smithi161 podman[69221]: 2024-04-24 23:29:30.919697512 +0000 UTC m=+5.707455767 container died b818a7a900acb1b416a2d20f4e0f522ffd4b524040b95963491ee3615ac4f474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T23:29:31.529 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:31 smithi161 podman[69221]: 2024-04-24 23:29:31.310679717 +0000 UTC m=+6.098437979 container cleanup b818a7a900acb1b416a2d20f4e0f522ffd4b524040b95963491ee3615ac4f474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T23:29:31.529 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:31 smithi161 bash[69221]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1 2024-04-24T23:29:31.858 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:31 smithi161 podman[69238]: 2024-04-24 23:29:31.528800162 +0000 UTC m=+0.601808528 container remove b818a7a900acb1b416a2d20f4e0f522ffd4b524040b95963491ee3615ac4f474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:29:32.282 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:31 smithi161 podman[69328]: 2024-04-24 23:29:31.867861118 +0000 UTC m=+0.018191251 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:29:32.282 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:32 smithi161 podman[69328]: 2024-04-24 23:29:32.013477979 +0000 UTC m=+0.163808154 container create 825c2937522e8f5deea478ad810ed004f49ec77d7475222a94606e6bc64b2a37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:29:32.608 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:32 smithi161 podman[69328]: 2024-04-24 23:29:32.282839021 +0000 UTC m=+0.433169155 container init 825c2937522e8f5deea478ad810ed004f49ec77d7475222a94606e6bc64b2a37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-24T23:29:32.608 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:32 smithi161 podman[69328]: 2024-04-24 23:29:32.286242431 +0000 UTC m=+0.436572564 container start 825c2937522e8f5deea478ad810ed004f49ec77d7475222a94606e6bc64b2a37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:29:32.608 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:32 smithi161 podman[69328]: 2024-04-24 23:29:32.391469768 +0000 UTC m=+0.541799908 container attach 825c2937522e8f5deea478ad810ed004f49ec77d7475222a94606e6bc64b2a37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-24T23:29:33.094 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 24 23:29:32 smithi161 podman[69328]: 2024-04-24 23:29:32.787147211 +0000 UTC m=+0.937477345 container died 825c2937522e8f5deea478ad810ed004f49ec77d7475222a94606e6bc64b2a37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:29:33.408 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T23:29:33.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:29:33.449 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T23:29:33.449 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T23:29:33.449 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.2 2024-04-24T23:29:33.761 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:33 smithi183 systemd[1]: Stopping Ceph osd.2 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:29:34.261 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:33 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2[38275]: 2024-04-24T23:29:33.763+0000 7f6025273700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:29:34.261 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:33 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2[38275]: 2024-04-24T23:29:33.763+0000 7f6025273700 -1 osd.2 111 *** Got signal Terminated *** 2024-04-24T23:29:34.261 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:33 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2[38275]: 2024-04-24T23:29:33.763+0000 7f6025273700 -1 osd.2 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:29:39.511 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:39 smithi183 podman[70763]: 2024-04-24 23:29:39.251439178 +0000 UTC m=+5.608598437 container died 3cc0b6e0952c5b4d78a0e61cb17fe160b200f3e66431fa2b2f82c67685377204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T23:29:39.916 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:39 smithi183 podman[70763]: 2024-04-24 23:29:39.639663268 +0000 UTC m=+5.996822534 container cleanup 3cc0b6e0952c5b4d78a0e61cb17fe160b200f3e66431fa2b2f82c67685377204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T23:29:39.916 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:39 smithi183 bash[70763]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2 2024-04-24T23:29:40.233 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:39 smithi183 podman[70782]: 2024-04-24 23:29:39.916502788 +0000 UTC m=+0.652527862 container remove 3cc0b6e0952c5b4d78a0e61cb17fe160b200f3e66431fa2b2f82c67685377204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:29:40.511 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:40 smithi183 podman[70871]: 2024-04-24 23:29:40.134611687 +0000 UTC m=+0.018100459 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:29:40.511 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:40 smithi183 podman[70871]: 2024-04-24 23:29:40.250860948 +0000 UTC m=+0.134349710 container create fa5f07e769742c4c1297d59686346a41020d7add9c6ec4c44c11062277446045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:29:41.011 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:40 smithi183 podman[70871]: 2024-04-24 23:29:40.520484381 +0000 UTC m=+0.403973152 container init fa5f07e769742c4c1297d59686346a41020d7add9c6ec4c44c11062277446045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:29:41.011 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:40 smithi183 podman[70871]: 2024-04-24 23:29:40.523668602 +0000 UTC m=+0.407157366 container start fa5f07e769742c4c1297d59686346a41020d7add9c6ec4c44c11062277446045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-24T23:29:41.011 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:40 smithi183 podman[70871]: 2024-04-24 23:29:40.654223177 +0000 UTC m=+0.537711951 container attach fa5f07e769742c4c1297d59686346a41020d7add9c6ec4c44c11062277446045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:29:41.351 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 24 23:29:41 smithi183 podman[70871]: 2024-04-24 23:29:41.073881703 +0000 UTC m=+0.957370467 container died fa5f07e769742c4c1297d59686346a41020d7add9c6ec4c44c11062277446045 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:29:41.739 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T23:29:41.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:29:41.777 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T23:29:41.777 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T23:29:41.777 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.3 2024-04-24T23:29:42.070 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:41 smithi102 systemd[1]: Stopping Ceph osd.3 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:29:42.570 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:42 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3[51263]: 2024-04-24T23:29:42.116+0000 7f0255209700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:29:42.571 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:42 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3[51263]: 2024-04-24T23:29:42.117+0000 7f0255209700 -1 osd.3 111 *** Got signal Terminated *** 2024-04-24T23:29:42.571 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:42 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3[51263]: 2024-04-24T23:29:42.117+0000 7f0255209700 -1 osd.3 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:29:48.196 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:47 smithi102 podman[132278]: 2024-04-24 23:29:47.889519113 +0000 UTC m=+5.920613467 container died 1c4e18cdd3c967c263b86dfce153ee9f46b67ef0d82105b2614af70d50b625af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-24T23:29:48.560 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:48 smithi102 podman[132278]: 2024-04-24 23:29:48.301173685 +0000 UTC m=+6.332268046 container cleanup 1c4e18cdd3c967c263b86dfce153ee9f46b67ef0d82105b2614af70d50b625af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T23:29:48.561 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:48 smithi102 bash[132278]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3 2024-04-24T23:29:48.820 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:48 smithi102 podman[132294]: 2024-04-24 23:29:48.560858227 +0000 UTC m=+0.669895217 container remove 1c4e18cdd3c967c263b86dfce153ee9f46b67ef0d82105b2614af70d50b625af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-24T23:29:49.306 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:48 smithi102 podman[132384]: 2024-04-24 23:29:48.879391243 +0000 UTC m=+0.022380610 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:29:49.306 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:48 smithi102 podman[132384]: 2024-04-24 23:29:48.987011798 +0000 UTC m=+0.130001184 container create fd28a5c64c5bd014dbbca9f51338d8bcf8b87615c6639c0c068ed9ba36f4fcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-24T23:29:49.306 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:49 smithi102 podman[132384]: 2024-04-24 23:29:49.214531059 +0000 UTC m=+0.357520403 container init fd28a5c64c5bd014dbbca9f51338d8bcf8b87615c6639c0c068ed9ba36f4fcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-24T23:29:49.306 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:49 smithi102 podman[132384]: 2024-04-24 23:29:49.217605315 +0000 UTC m=+0.360594657 container start fd28a5c64c5bd014dbbca9f51338d8bcf8b87615c6639c0c068ed9ba36f4fcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T23:29:49.306 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:49 smithi102 podman[132384]: 2024-04-24 23:29:49.306299083 +0000 UTC m=+0.449288423 container attach fd28a5c64c5bd014dbbca9f51338d8bcf8b87615c6639c0c068ed9ba36f4fcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:29:49.571 INFO:journalctl@ceph.osd.3.smithi102.stdout:Apr 24 23:29:49 smithi102 podman[132384]: 2024-04-24 23:29:49.505569748 +0000 UTC m=+0.648559093 container died fd28a5c64c5bd014dbbca9f51338d8bcf8b87615c6639c0c068ed9ba36f4fcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T23:29:50.204 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T23:29:50.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:29:50.244 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T23:29:50.244 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T23:29:50.244 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.4 2024-04-24T23:29:50.539 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:50 smithi161 systemd[1]: Stopping Ceph osd.4 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:29:50.805 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:50 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4[44225]: 2024-04-24T23:29:50.538+0000 7ff543416700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:29:50.805 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:50 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4[44225]: 2024-04-24T23:29:50.538+0000 7ff543416700 -1 osd.4 111 *** Got signal Terminated *** 2024-04-24T23:29:50.805 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:50 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4[44225]: 2024-04-24T23:29:50.538+0000 7ff543416700 -1 osd.4 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:29:56.459 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:56 smithi161 podman[69433]: 2024-04-24 23:29:56.162571418 +0000 UTC m=+5.727570695 container died 102325cc59333e0e7ad0b0924a785012d4d6955c6454c2e4924a912ab2e6259e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T23:29:56.824 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:56 smithi161 podman[69433]: 2024-04-24 23:29:56.55659182 +0000 UTC m=+6.121591092 container cleanup 102325cc59333e0e7ad0b0924a785012d4d6955c6454c2e4924a912ab2e6259e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-24T23:29:56.824 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:56 smithi161 bash[69433]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4 2024-04-24T23:29:57.108 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:56 smithi161 podman[69462]: 2024-04-24 23:29:56.824627401 +0000 UTC m=+0.658293861 container remove 102325cc59333e0e7ad0b0924a785012d4d6955c6454c2e4924a912ab2e6259e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:29:57.453 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:57 smithi161 podman[69551]: 2024-04-24 23:29:57.072686904 +0000 UTC m=+0.018003545 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:29:57.453 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:57 smithi161 podman[69551]: 2024-04-24 23:29:57.19214177 +0000 UTC m=+0.137458410 container create 262ea6f92130ac7be6a70d7347891b247a097718ad7fa47f8ae08762ff5f360e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T23:29:57.453 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:57 smithi161 podman[69551]: 2024-04-24 23:29:57.453052076 +0000 UTC m=+0.398368720 container init 262ea6f92130ac7be6a70d7347891b247a097718ad7fa47f8ae08762ff5f360e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T23:29:57.748 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:57 smithi161 podman[69551]: 2024-04-24 23:29:57.456114277 +0000 UTC m=+0.401430920 container start 262ea6f92130ac7be6a70d7347891b247a097718ad7fa47f8ae08762ff5f360e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T23:29:57.748 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:57 smithi161 podman[69551]: 2024-04-24 23:29:57.553145684 +0000 UTC m=+0.498462331 container attach 262ea6f92130ac7be6a70d7347891b247a097718ad7fa47f8ae08762ff5f360e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-24T23:29:58.047 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 24 23:29:57 smithi161 podman[69551]: 2024-04-24 23:29:57.748816755 +0000 UTC m=+0.694133398 container died 262ea6f92130ac7be6a70d7347891b247a097718ad7fa47f8ae08762ff5f360e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T23:29:58.363 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T23:29:58.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:29:58.402 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T23:29:58.402 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T23:29:58.402 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.5 2024-04-24T23:29:58.702 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:29:58 smithi183 systemd[1]: Stopping Ceph osd.5 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:29:59.011 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:29:58 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5[45242]: 2024-04-24T23:29:58.701+0000 7f12837c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:29:59.011 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:29:58 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5[45242]: 2024-04-24T23:29:58.701+0000 7f12837c6700 -1 osd.5 111 *** Got signal Terminated *** 2024-04-24T23:29:59.011 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:29:58 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5[45242]: 2024-04-24T23:29:58.701+0000 7f12837c6700 -1 osd.5 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:30:04.669 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:04 smithi183 podman[70988]: 2024-04-24 23:30:04.329167564 +0000 UTC m=+5.729646440 container died 8b7bc6e5798272a07c10eb296141d36d99165a37eb3861f65a525a2f457a81d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:30:05.011 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:04 smithi183 podman[70988]: 2024-04-24 23:30:04.669481203 +0000 UTC m=+6.069960092 container cleanup 8b7bc6e5798272a07c10eb296141d36d99165a37eb3861f65a525a2f457a81d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T23:30:05.011 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:04 smithi183 bash[70988]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5 2024-04-24T23:30:05.011 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:04 smithi183 podman[71007]: 2024-04-24 23:30:04.912468669 +0000 UTC m=+0.581058383 container remove 8b7bc6e5798272a07c10eb296141d36d99165a37eb3861f65a525a2f457a81d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:30:05.617 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:05 smithi183 podman[71096]: 2024-04-24 23:30:05.220597758 +0000 UTC m=+0.018224511 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:30:05.617 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:05 smithi183 podman[71096]: 2024-04-24 23:30:05.338675642 +0000 UTC m=+0.136302394 container create 902f3d914ced7bb7481ba842da717c15f9a522a0ba41d1e6a9db04e9a58709ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T23:30:05.912 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:05 smithi183 podman[71096]: 2024-04-24 23:30:05.617459096 +0000 UTC m=+0.415085849 container init 902f3d914ced7bb7481ba842da717c15f9a522a0ba41d1e6a9db04e9a58709ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-24T23:30:05.912 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:05 smithi183 podman[71096]: 2024-04-24 23:30:05.620736155 +0000 UTC m=+0.418362909 container start 902f3d914ced7bb7481ba842da717c15f9a522a0ba41d1e6a9db04e9a58709ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T23:30:05.913 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:05 smithi183 podman[71096]: 2024-04-24 23:30:05.717612312 +0000 UTC m=+0.515239066 container attach 902f3d914ced7bb7481ba842da717c15f9a522a0ba41d1e6a9db04e9a58709ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:30:06.261 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 24 23:30:05 smithi183 podman[71096]: 2024-04-24 23:30:05.912806238 +0000 UTC m=+0.710433000 container died 902f3d914ced7bb7481ba842da717c15f9a522a0ba41d1e6a9db04e9a58709ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:30:06.596 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T23:30:06.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:30:06.641 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T23:30:06.641 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-24T23:30:06.641 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.6 2024-04-24T23:30:06.938 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:06 smithi102 systemd[1]: Stopping Ceph osd.6 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:30:07.320 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:06 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6[57809]: 2024-04-24T23:30:06.938+0000 7fa73a5a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:30:07.321 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:06 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6[57809]: 2024-04-24T23:30:06.938+0000 7fa73a5a9700 -1 osd.6 111 *** Got signal Terminated *** 2024-04-24T23:30:07.321 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:06 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6[57809]: 2024-04-24T23:30:06.938+0000 7fa73a5a9700 -1 osd.6 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:30:12.738 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:12 smithi102 podman[132490]: 2024-04-24 23:30:12.464389899 +0000 UTC m=+5.630443645 container died 019ffb9edd4f84c1fe7fdd80d1c306ce593c3105e51c2c4b3d3b9f7e110fdd75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-24T23:30:13.070 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:12 smithi102 podman[132490]: 2024-04-24 23:30:12.820882932 +0000 UTC m=+5.986936676 container cleanup 019ffb9edd4f84c1fe7fdd80d1c306ce593c3105e51c2c4b3d3b9f7e110fdd75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-24T23:30:13.071 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:12 smithi102 bash[132490]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6 2024-04-24T23:30:13.483 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:13 smithi102 podman[132507]: 2024-04-24 23:30:13.138911805 +0000 UTC m=+0.675108130 container remove 019ffb9edd4f84c1fe7fdd80d1c306ce593c3105e51c2c4b3d3b9f7e110fdd75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T23:30:13.821 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:13 smithi102 podman[132600]: 2024-04-24 23:30:13.386592249 +0000 UTC m=+0.018168814 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:30:13.821 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:13 smithi102 podman[132600]: 2024-04-24 23:30:13.498074081 +0000 UTC m=+0.129650630 container create 765cddd91e552f09f5d1ff501e364dbe323fcd7d586116a45743b8ad7f4c1c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:30:13.821 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:13 smithi102 podman[132600]: 2024-04-24 23:30:13.717351924 +0000 UTC m=+0.348928476 container init 765cddd91e552f09f5d1ff501e364dbe323fcd7d586116a45743b8ad7f4c1c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T23:30:13.821 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:13 smithi102 podman[132600]: 2024-04-24 23:30:13.720419116 +0000 UTC m=+0.351995668 container start 765cddd91e552f09f5d1ff501e364dbe323fcd7d586116a45743b8ad7f4c1c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:30:14.275 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:13 smithi102 podman[132600]: 2024-04-24 23:30:13.83388514 +0000 UTC m=+0.465461708 container attach 765cddd91e552f09f5d1ff501e364dbe323fcd7d586116a45743b8ad7f4c1c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T23:30:14.275 INFO:journalctl@ceph.osd.6.smithi102.stdout:Apr 24 23:30:14 smithi102 podman[132600]: 2024-04-24 23:30:14.006410816 +0000 UTC m=+0.637987366 container died 765cddd91e552f09f5d1ff501e364dbe323fcd7d586116a45743b8ad7f4c1c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T23:30:14.701 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.6.service' 2024-04-24T23:30:14.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:30:14.739 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-24T23:30:14.739 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-24T23:30:14.740 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.7 2024-04-24T23:30:15.042 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:14 smithi161 systemd[1]: Stopping Ceph osd.7 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:30:15.358 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:15 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7[50762]: 2024-04-24T23:30:15.041+0000 7f0b51c90700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:30:15.358 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:15 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7[50762]: 2024-04-24T23:30:15.041+0000 7f0b51c90700 -1 osd.7 111 *** Got signal Terminated *** 2024-04-24T23:30:15.358 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:15 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7[50762]: 2024-04-24T23:30:15.041+0000 7f0b51c90700 -1 osd.7 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:30:20.975 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:20 smithi161 podman[69657]: 2024-04-24 23:30:20.670182358 +0000 UTC m=+5.741826488 container died 36b262d881b1af186acce43196aefa4bec56b889e0020bc5ca7b2d8e1ddd414c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T23:30:21.334 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:21 smithi161 podman[69657]: 2024-04-24 23:30:21.091767778 +0000 UTC m=+6.163411908 container cleanup 36b262d881b1af186acce43196aefa4bec56b889e0020bc5ca7b2d8e1ddd414c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T23:30:21.334 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:21 smithi161 bash[69657]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7 2024-04-24T23:30:21.608 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:21 smithi161 podman[69674]: 2024-04-24 23:30:21.334773366 +0000 UTC m=+0.663398163 container remove 36b262d881b1af186acce43196aefa4bec56b889e0020bc5ca7b2d8e1ddd414c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T23:30:22.108 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:21 smithi161 podman[69777]: 2024-04-24 23:30:21.785557346 +0000 UTC m=+0.114355020 container create cb5a64d86899a00fdccb1d1f391b81633b09c2dfad21e43116c3af99d6462cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T23:30:22.109 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:21 smithi161 podman[69777]: 2024-04-24 23:30:21.694191817 +0000 UTC m=+0.022989509 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:30:22.109 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:22 smithi161 podman[69777]: 2024-04-24 23:30:22.021684657 +0000 UTC m=+0.350482332 container init cb5a64d86899a00fdccb1d1f391b81633b09c2dfad21e43116c3af99d6462cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T23:30:22.109 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:22 smithi161 podman[69777]: 2024-04-24 23:30:22.025206135 +0000 UTC m=+0.354003810 container start cb5a64d86899a00fdccb1d1f391b81633b09c2dfad21e43116c3af99d6462cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T23:30:22.608 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:22 smithi161 podman[69777]: 2024-04-24 23:30:22.148575977 +0000 UTC m=+0.477373665 container attach cb5a64d86899a00fdccb1d1f391b81633b09c2dfad21e43116c3af99d6462cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T23:30:22.608 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 24 23:30:22 smithi161 podman[69777]: 2024-04-24 23:30:22.318040277 +0000 UTC m=+0.646837967 container died cb5a64d86899a00fdccb1d1f391b81633b09c2dfad21e43116c3af99d6462cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T23:30:22.950 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:22 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:22.710+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:22.973 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.7.service' 2024-04-24T23:30:23.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:30:23.012 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-24T23:30:23.012 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-24T23:30:23.013 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.8 2024-04-24T23:30:23.326 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:23 smithi183 systemd[1]: Stopping Ceph osd.8 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:30:23.651 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:23 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8[51784]: 2024-04-24T23:30:23.325+0000 7fe1155bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:30:23.651 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:23 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8[51784]: 2024-04-24T23:30:23.325+0000 7fe1155bc700 -1 osd.8 111 *** Got signal Terminated *** 2024-04-24T23:30:23.651 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:23 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8[51784]: 2024-04-24T23:30:23.325+0000 7fe1155bc700 -1 osd.8 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:30:24.011 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:23 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:23.871+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:24.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:23 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:23.676+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:25.071 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:24 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:24.703+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:25.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:24 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:24.722+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:25.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:24 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:24.918+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:26.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:25 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:25.662+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:26.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:25 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:25.731+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:26.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:25 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:25.917+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:27.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:26 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:26.678+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:27.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:26 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:26.699+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:27.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:26 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:26.958+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:28.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:27 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:27.706+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:28.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:27 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:27.735+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:28.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:27 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:27.929+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:29.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:28 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:28.667+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:29.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:28 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:28.718+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:29.261 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:29 smithi183 podman[71199]: 2024-04-24 23:30:29.107615875 +0000 UTC m=+5.902984412 container died 3b3dd2989d75c8f3e029729e21c39c1d5a8fec20327556bb76c9b4e31f889e42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T23:30:29.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:28 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:28.935+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:29.761 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:29 smithi183 podman[71199]: 2024-04-24 23:30:29.517170575 +0000 UTC m=+6.312539103 container cleanup 3b3dd2989d75c8f3e029729e21c39c1d5a8fec20327556bb76c9b4e31f889e42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-24T23:30:29.761 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:29 smithi183 bash[71199]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8 2024-04-24T23:30:30.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:29 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:29.699+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:30.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:29 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:29.697+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:30.177 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:29 smithi183 podman[71227]: 2024-04-24 23:30:29.777112803 +0000 UTC m=+0.666476724 container remove 3b3dd2989d75c8f3e029729e21c39c1d5a8fec20327556bb76c9b4e31f889e42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-24T23:30:30.177 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:29 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:29.906+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:30.511 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:30 smithi183 podman[71313]: 2024-04-24 23:30:30.079651634 +0000 UTC m=+0.018162287 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:30:30.511 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:30 smithi183 podman[71313]: 2024-04-24 23:30:30.228285949 +0000 UTC m=+0.166796578 container create 999ab61b2f3d8c6e5c4e744cf4d56f2de428d36e90d093fd1fddc3857404e571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:30:30.814 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:30 smithi183 podman[71313]: 2024-04-24 23:30:30.522690559 +0000 UTC m=+0.461201192 container init 999ab61b2f3d8c6e5c4e744cf4d56f2de428d36e90d093fd1fddc3857404e571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-24T23:30:30.814 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:30 smithi183 podman[71313]: 2024-04-24 23:30:30.525827728 +0000 UTC m=+0.464338359 container start 999ab61b2f3d8c6e5c4e744cf4d56f2de428d36e90d093fd1fddc3857404e571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T23:30:30.815 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:30 smithi183 podman[71313]: 2024-04-24 23:30:30.597609101 +0000 UTC m=+0.536119749 container attach 999ab61b2f3d8c6e5c4e744cf4d56f2de428d36e90d093fd1fddc3857404e571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-24T23:30:31.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:30 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:30.652+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:31.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:30 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:30.727+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:31.167 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:30 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:30.944+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:31.167 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 24 23:30:30 smithi183 podman[71313]: 2024-04-24 23:30:30.814897664 +0000 UTC m=+0.753408298 container died 999ab61b2f3d8c6e5c4e744cf4d56f2de428d36e90d093fd1fddc3857404e571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-24T23:30:31.461 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.8.service' 2024-04-24T23:30:31.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:30:31.501 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-24T23:30:31.501 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-24T23:30:31.501 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.9 2024-04-24T23:30:31.821 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:31 smithi102 systemd[1]: Stopping Ceph osd.9 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:30:31.821 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:31 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:31.692+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:32.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:31 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:31.690+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:32.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:31 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:31.955+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:32.321 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:31 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:31.823+0000 7fa895953700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:30:32.321 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:31 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:31.823+0000 7fa895953700 -1 osd.9 111 *** Got signal Terminated *** 2024-04-24T23:30:32.321 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:31 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:31.823+0000 7fa895953700 -1 osd.9 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:30:33.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:32 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:32.661+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:33.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:32 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:32.673+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:33.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:32 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:32.673+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:33.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:32 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:32.963+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:34.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:33 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:33.657+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:34.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:33 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:33.657+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.886320+0000 front 2024-04-24T23:29:29.886305+0000 (oldest deadline 2024-04-24T23:30:33.386396+0000) 2024-04-24T23:30:34.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:33 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:33.698+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:34.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:33 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:33.698+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:34.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:33 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:33.934+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:34.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:33 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:33.934+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:35.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:34 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:34.653+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:35.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:34 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:34.653+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.886320+0000 front 2024-04-24T23:29:29.886305+0000 (oldest deadline 2024-04-24T23:30:33.386396+0000) 2024-04-24T23:30:35.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:34 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:34.730+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:35.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:34 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:34.730+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:35.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:34 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:34.931+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:35.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:34 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:34.931+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:36.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:35 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:35.656+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:36.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:35 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:35.656+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.886320+0000 front 2024-04-24T23:29:29.886305+0000 (oldest deadline 2024-04-24T23:30:33.386396+0000) 2024-04-24T23:30:36.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:35 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:35.707+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:36.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:35 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:35.707+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:36.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:35 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:35.972+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:36.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:35 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:35.972+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:37.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:36 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:36.694+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:21.884090+0000 front 2024-04-24T23:29:21.884118+0000 (oldest deadline 2024-04-24T23:30:24.184203+0000) 2024-04-24T23:30:37.071 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:36 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9[64590]: 2024-04-24T23:30:36.694+0000 7fa88df3f700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.886320+0000 front 2024-04-24T23:29:29.886305+0000 (oldest deadline 2024-04-24T23:30:33.386396+0000) 2024-04-24T23:30:37.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:36 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:36.679+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:37.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:36 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:36.679+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:37.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:36 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:36.965+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:37.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:36 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:36.965+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:37.733 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:37 smithi102 podman[132718]: 2024-04-24 23:30:37.446194595 +0000 UTC m=+5.747933582 container died ade063394ff463401e3be44b1be7d083a8468e7de205c4a278ecc456cf71773f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T23:30:38.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:37 smithi102 podman[132718]: 2024-04-24 23:30:37.840433191 +0000 UTC m=+6.142172183 container cleanup ade063394ff463401e3be44b1be7d083a8468e7de205c4a278ecc456cf71773f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:30:38.071 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:37 smithi102 bash[132718]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9 2024-04-24T23:30:38.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:37 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:37.711+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:38.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:37 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:37.711+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:38.262 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:37 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:37.983+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:38.262 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:37 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:37.983+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:38.459 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:38 smithi102 podman[132735]: 2024-04-24 23:30:38.166752206 +0000 UTC m=+0.717508057 container remove ade063394ff463401e3be44b1be7d083a8468e7de205c4a278ecc456cf71773f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T23:30:38.744 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:38 smithi102 podman[132826]: 2024-04-24 23:30:38.362277811 +0000 UTC m=+0.018096876 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:30:38.744 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:38 smithi102 podman[132826]: 2024-04-24 23:30:38.467822845 +0000 UTC m=+0.123641956 container create b4cf39b3483a774c4e6eb5b4e957748fd275d0e89f9dd87b8a25dc0399e728de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:30:38.745 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:38 smithi102 podman[132826]: 2024-04-24 23:30:38.662266877 +0000 UTC m=+0.318085922 container init b4cf39b3483a774c4e6eb5b4e957748fd275d0e89f9dd87b8a25dc0399e728de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:30:38.745 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:38 smithi102 podman[132826]: 2024-04-24 23:30:38.665758065 +0000 UTC m=+0.321577110 container start b4cf39b3483a774c4e6eb5b4e957748fd275d0e89f9dd87b8a25dc0399e728de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-24T23:30:38.745 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:38 smithi102 podman[132826]: 2024-04-24 23:30:38.744953481 +0000 UTC m=+0.400772534 container attach b4cf39b3483a774c4e6eb5b4e957748fd275d0e89f9dd87b8a25dc0399e728de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:30:39.070 INFO:journalctl@ceph.osd.9.smithi102.stdout:Apr 24 23:30:38 smithi102 podman[132826]: 2024-04-24 23:30:38.958461582 +0000 UTC m=+0.614280632 container died b4cf39b3483a774c4e6eb5b4e957748fd275d0e89f9dd87b8a25dc0399e728de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, ceph=True) 2024-04-24T23:30:39.109 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:38 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:38.710+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:39.109 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:38 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:38.710+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:39.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:39 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:39.007+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:39.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:39 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:39.008+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:39.461 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.9.service' 2024-04-24T23:30:39.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:30:39.498 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-24T23:30:39.498 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-24T23:30:39.498 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.10 2024-04-24T23:30:39.785 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:39 smithi161 systemd[1]: Stopping Ceph osd.10 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:30:39.785 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:39 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:39.685+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:39.786 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:39 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:39.685+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:39.786 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:39 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:39.784+0000 7f05e7a25700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:30:39.786 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:39 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:39.784+0000 7f05e7a25700 -1 osd.10 111 *** Got signal Terminated *** 2024-04-24T23:30:39.786 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:39 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:39.784+0000 7f05e7a25700 -1 osd.10 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:30:40.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:39 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:39.971+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:40.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:39 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:39.971+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:40.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:39 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:39.971+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:41.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:40 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:40.714+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:41.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:40 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:40.714+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:41.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:40 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:40.714+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:37.130353+0000 front 2024-04-24T23:29:37.130520+0000 (oldest deadline 2024-04-24T23:30:40.030160+0000) 2024-04-24T23:30:41.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:40 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:40.988+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:41.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:40 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:40.988+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:41.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:40 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:40.988+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:42.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:41 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:41.689+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:42.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:41 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:41.689+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:42.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:41 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:41.689+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:37.130353+0000 front 2024-04-24T23:29:37.130520+0000 (oldest deadline 2024-04-24T23:30:40.030160+0000) 2024-04-24T23:30:42.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:41 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:41.951+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:42.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:41 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:41.951+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:42.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:41 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:41.951+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:43.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:42 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:42.697+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:43.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:42 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:42.697+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:43.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:42 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:42.697+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:37.130353+0000 front 2024-04-24T23:29:37.130520+0000 (oldest deadline 2024-04-24T23:30:40.030160+0000) 2024-04-24T23:30:43.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:42 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:42.973+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:43.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:42 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:42.973+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:43.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:42 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:42.973+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:44.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:43 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:43.740+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:44.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:43 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:43.740+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:44.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:43 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:43.740+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:37.130353+0000 front 2024-04-24T23:29:37.130520+0000 (oldest deadline 2024-04-24T23:30:40.030160+0000) 2024-04-24T23:30:44.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:43 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:43.956+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:44.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:43 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:43.956+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:44.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:43 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:43.956+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:45.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:44 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:44.733+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:19.628196+0000 front 2024-04-24T23:29:19.627942+0000 (oldest deadline 2024-04-24T23:30:22.527897+0000) 2024-04-24T23:30:45.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:44 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:44.733+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:27.729290+0000 front 2024-04-24T23:29:27.729418+0000 (oldest deadline 2024-04-24T23:30:32.429418+0000) 2024-04-24T23:30:45.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:44 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-24T23:30:44.733+0000 7f05e4019700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:37.130353+0000 front 2024-04-24T23:29:37.130520+0000 (oldest deadline 2024-04-24T23:30:40.030160+0000) 2024-04-24T23:30:45.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:44 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:44.940+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:45.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:44 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:44.940+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:45.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:44 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:44.940+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:45.858 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:45 smithi161 podman[69884]: 2024-04-24 23:30:45.588216645 +0000 UTC m=+5.899526152 container died fa73740dec1096d6743ad8c22d17d89cb89d8e4d33ca8205b6a4cf12297fe19e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:30:46.230 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:45 smithi161 podman[69884]: 2024-04-24 23:30:45.942828897 +0000 UTC m=+6.254138411 container cleanup fa73740dec1096d6743ad8c22d17d89cb89d8e4d33ca8205b6a4cf12297fe19e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-04-24T23:30:46.230 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:45 smithi161 bash[69884]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10 2024-04-24T23:30:46.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:45 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:45.967+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:46.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:45 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:45.967+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:46.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:45 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:45.967+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:46.553 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:46 smithi161 podman[69901]: 2024-04-24 23:30:46.227147784 +0000 UTC m=+0.635374230 container remove fa73740dec1096d6743ad8c22d17d89cb89d8e4d33ca8205b6a4cf12297fe19e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424) 2024-04-24T23:30:46.855 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:46 smithi161 podman[69988]: 2024-04-24 23:30:46.456677125 +0000 UTC m=+0.017831495 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:30:46.855 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:46 smithi161 podman[69988]: 2024-04-24 23:30:46.561485041 +0000 UTC m=+0.122639380 container create e35d34a4d3d3aea27862bbf76a3d9dbfb2c6c235251cf116741f56751bb44dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:30:46.855 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:46 smithi161 podman[69988]: 2024-04-24 23:30:46.780501554 +0000 UTC m=+0.341655901 container init e35d34a4d3d3aea27862bbf76a3d9dbfb2c6c235251cf116741f56751bb44dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:30:46.855 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:46 smithi161 podman[69988]: 2024-04-24 23:30:46.783626904 +0000 UTC m=+0.344781248 container start e35d34a4d3d3aea27862bbf76a3d9dbfb2c6c235251cf116741f56751bb44dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:30:46.855 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:46 smithi161 podman[69988]: 2024-04-24 23:30:46.85533026 +0000 UTC m=+0.416484599 container attach e35d34a4d3d3aea27862bbf76a3d9dbfb2c6c235251cf116741f56751bb44dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-24T23:30:47.108 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:47 smithi161 podman[69988]: 2024-04-24 23:30:47.077188004 +0000 UTC m=+0.638342347 container died e35d34a4d3d3aea27862bbf76a3d9dbfb2c6c235251cf116741f56751bb44dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:30:47.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:46 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:46.991+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:47.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:46 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:46.991+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:47.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:46 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:46.991+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:47.607 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.10.service' 2024-04-24T23:30:47.608 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:47 smithi161 podman[70014]: 2024-04-24 23:30:47.591656264 +0000 UTC m=+0.509942800 container remove e35d34a4d3d3aea27862bbf76a3d9dbfb2c6c235251cf116741f56751bb44dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T23:30:47.608 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:47 smithi161 systemd[1]: ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-24T23:30:47.608 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:47 smithi161 systemd[1]: Stopped Ceph osd.10 for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:30:47.608 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 24 23:30:47 smithi161 systemd[1]: ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.10.service: Consumed 7min 11.656s CPU time. 2024-04-24T23:30:47.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:30:47.645 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-24T23:30:47.645 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-24T23:30:47.645 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.11 2024-04-24T23:30:47.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:47 smithi183 systemd[1]: Stopping Ceph osd.11 for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:30:47.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:47 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:47.951+0000 7fd58ac47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:30:48.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:47 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:47.951+0000 7fd58ac47700 -1 osd.11 111 *** Got signal Terminated *** 2024-04-24T23:30:48.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:47 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:47.951+0000 7fd58ac47700 -1 osd.11 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:30:48.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:47 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:47.992+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:48.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:47 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:47.992+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:48.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:47 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:47.992+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:49.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:48 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:48.963+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:49.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:48 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:48.963+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:49.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:48 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:48.963+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:50.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:50 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:50.006+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:50.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:50 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:50.006+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:50.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:50 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:50.006+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:50.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:50 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:50.006+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6814 osd.3 since back 2024-04-24T23:29:45.523848+0000 front 2024-04-24T23:29:45.523813+0000 (oldest deadline 2024-04-24T23:30:49.023718+0000) 2024-04-24T23:30:51.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:50 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:50.996+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:51.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:50 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:50.996+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:51.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:50 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:50.996+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:51.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:50 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:50.996+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6814 osd.3 since back 2024-04-24T23:29:45.523848+0000 front 2024-04-24T23:29:45.523813+0000 (oldest deadline 2024-04-24T23:30:49.023718+0000) 2024-04-24T23:30:52.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:51 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:51.956+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-04-24T23:29:17.519421+0000 front 2024-04-24T23:29:17.519450+0000 (oldest deadline 2024-04-24T23:30:23.419503+0000) 2024-04-24T23:30:52.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:51 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:51.956+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-24T23:29:29.320910+0000 front 2024-04-24T23:29:29.320781+0000 (oldest deadline 2024-04-24T23:30:33.321306+0000) 2024-04-24T23:30:52.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:51 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:51.956+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-24T23:29:35.622245+0000 front 2024-04-24T23:29:35.622281+0000 (oldest deadline 2024-04-24T23:30:39.722393+0000) 2024-04-24T23:30:52.261 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:51 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11[58388]: 2024-04-24T23:30:51.956+0000 7fd583233700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.102:6814 osd.3 since back 2024-04-24T23:29:45.523848+0000 front 2024-04-24T23:29:45.523813+0000 (oldest deadline 2024-04-24T23:30:49.023718+0000) 2024-04-24T23:30:53.886 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:53 smithi183 podman[71423]: 2024-04-24 23:30:53.62470161 +0000 UTC m=+5.788271964 container died a36b90069a5dd972acab556b867095943814538ad4e05eae512f6a6d8e9843ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11, ceph=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:30:54.237 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:53 smithi183 podman[71423]: 2024-04-24 23:30:53.986092605 +0000 UTC m=+6.149662956 container cleanup a36b90069a5dd972acab556b867095943814538ad4e05eae512f6a6d8e9843ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T23:30:54.237 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:53 smithi183 bash[71423]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11 2024-04-24T23:30:54.511 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:54 smithi183 podman[71441]: 2024-04-24 23:30:54.237196741 +0000 UTC m=+0.612374098 container remove a36b90069a5dd972acab556b867095943814538ad4e05eae512f6a6d8e9843ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:30:54.801 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:54 smithi183 podman[71534]: 2024-04-24 23:30:54.453051135 +0000 UTC m=+0.018173652 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:30:54.801 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:54 smithi183 podman[71534]: 2024-04-24 23:30:54.563039824 +0000 UTC m=+0.128162323 container create 340cd6e7f4f0152dec459a66ad9aa4d7f117e270770c0b7f4e0f83e5979c4e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:30:54.801 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:54 smithi183 podman[71534]: 2024-04-24 23:30:54.798696613 +0000 UTC m=+0.363819113 container init 340cd6e7f4f0152dec459a66ad9aa4d7f117e270770c0b7f4e0f83e5979c4e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:30:54.801 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:54 smithi183 podman[71534]: 2024-04-24 23:30:54.80167337 +0000 UTC m=+0.366795870 container start 340cd6e7f4f0152dec459a66ad9aa4d7f117e270770c0b7f4e0f83e5979c4e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:30:55.092 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:54 smithi183 podman[71534]: 2024-04-24 23:30:54.890471692 +0000 UTC m=+0.455594206 container attach 340cd6e7f4f0152dec459a66ad9aa4d7f117e270770c0b7f4e0f83e5979c4e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:30:55.366 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 24 23:30:55 smithi183 podman[71534]: 2024-04-24 23:30:55.093441633 +0000 UTC m=+0.658564130 container died 340cd6e7f4f0152dec459a66ad9aa4d7f117e270770c0b7f4e0f83e5979c4e7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-24T23:30:55.688 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@osd.11.service' 2024-04-24T23:30:55.726 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:30:55.726 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-24T23:30:55.726 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-24T23:30:55.726 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.a 2024-04-24T23:30:56.024 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:30:55 smithi102 systemd[1]: Stopping Ceph mds.a for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:30:56.278 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:30:56 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-a[71350]: 2024-04-24T23:30:56.024+0000 7f4eb7812700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:30:56.278 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:30:56 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-a[71350]: 2024-04-24T23:30:56.024+0000 7f4eb7812700 -1 mds.a *** got signal Terminated *** 2024-04-24T23:31:00.320 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:31:00 smithi102 podman[132933]: 2024-04-24 23:31:00.060371259 +0000 UTC m=+4.139868524 container died 330491a062d525cc0c430fe410490e5a270ddf457fe6417ffa074c9758833140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T23:31:00.683 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:31:00 smithi102 podman[132933]: 2024-04-24 23:31:00.432907973 +0000 UTC m=+4.512405250 container cleanup 330491a062d525cc0c430fe410490e5a270ddf457fe6417ffa074c9758833140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T23:31:00.684 INFO:journalctl@ceph.mds.a.smithi102.stdout:Apr 24 23:31:00 smithi102 bash[132933]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-a 2024-04-24T23:31:00.823 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.a.service' 2024-04-24T23:31:00.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:00.858 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-24T23:31:00.858 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-24T23:31:00.858 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.d 2024-04-24T23:31:01.212 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:00 smithi102 systemd[1]: Stopping Ceph mds.d for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:01.570 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d[70344]: 2024-04-24T23:31:01.212+0000 7f02b5bde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:01.571 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:01 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d[70344]: 2024-04-24T23:31:01.212+0000 7f02b5bde700 -1 mds.d *** got signal Terminated *** 2024-04-24T23:31:04.329 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:04 smithi102 podman[133068]: 2024-04-24 23:31:04.072598041 +0000 UTC m=+2.988973430 container died 2dcddcfd0cb40f24ee5f17b081c559dc881138bf3344ac70ff02fb807a2f8d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T23:31:04.587 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:04 smithi102 podman[133068]: 2024-04-24 23:31:04.403331038 +0000 UTC m=+3.319706435 container cleanup 2dcddcfd0cb40f24ee5f17b081c559dc881138bf3344ac70ff02fb807a2f8d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:31:04.587 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:04 smithi102 bash[133068]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d 2024-04-24T23:31:04.830 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.d.service' 2024-04-24T23:31:04.850 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:04 smithi102 podman[133081]: 2024-04-24 23:31:04.587768906 +0000 UTC m=+0.510137237 container remove 2dcddcfd0cb40f24ee5f17b081c559dc881138bf3344ac70ff02fb807a2f8d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T23:31:04.851 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:04 smithi102 systemd[1]: ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-24T23:31:04.851 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:04 smithi102 systemd[1]: Stopped Ceph mds.d for 97eecd66-028e-11ef-bc93-c7b262605968. 2024-04-24T23:31:04.851 INFO:journalctl@ceph.mds.d.smithi102.stdout:Apr 24 23:31:04 smithi102 systemd[1]: ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.d.service: Consumed 1.091s CPU time. 2024-04-24T23:31:04.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:04.863 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-24T23:31:04.863 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-24T23:31:04.864 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.g 2024-04-24T23:31:05.207 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:31:04 smithi102 systemd[1]: Stopping Ceph mds.g for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:05.570 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:31:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-g[70953]: 2024-04-24T23:31:05.206+0000 7f0cb9826700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:05.570 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:31:05 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-g[70953]: 2024-04-24T23:31:05.206+0000 7f0cb9826700 -1 mds.g *** got signal Terminated *** 2024-04-24T23:31:09.491 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:31:09 smithi102 podman[133190]: 2024-04-24 23:31:09.215579631 +0000 UTC m=+4.137265282 container died 3f1a557f2ec0e174e60959d71b98f937b08db66dc06e2a80f1b043cfd23356b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T23:31:09.820 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:31:09 smithi102 podman[133190]: 2024-04-24 23:31:09.573606699 +0000 UTC m=+4.495292351 container cleanup 3f1a557f2ec0e174e60959d71b98f937b08db66dc06e2a80f1b043cfd23356b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T23:31:09.821 INFO:journalctl@ceph.mds.g.smithi102.stdout:Apr 24 23:31:09 smithi102 bash[133190]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-g 2024-04-24T23:31:10.006 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.g.service' 2024-04-24T23:31:10.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:10.038 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-24T23:31:10.039 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-24T23:31:10.039 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.j 2024-04-24T23:31:10.571 INFO:journalctl@ceph.mds.j.smithi102.stdout:Apr 24 23:31:10 smithi102 systemd[1]: Stopping Ceph mds.j for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:10.571 INFO:journalctl@ceph.mds.j.smithi102.stdout:Apr 24 23:31:10 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-j[69993]: 2024-04-24T23:31:10.359+0000 7f67452d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:10.571 INFO:journalctl@ceph.mds.j.smithi102.stdout:Apr 24 23:31:10 smithi102 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-j[69993]: 2024-04-24T23:31:10.359+0000 7f67452d2700 -1 mds.j *** got signal Terminated *** 2024-04-24T23:31:13.443 INFO:journalctl@ceph.mds.j.smithi102.stdout:Apr 24 23:31:13 smithi102 podman[133310]: 2024-04-24 23:31:13.147096021 +0000 UTC m=+2.895694883 container died 4ae03cbd359f9d19018af8ec9128126154c77c6628fb42dd3f5ae48ba6650f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:31:13.769 INFO:journalctl@ceph.mds.j.smithi102.stdout:Apr 24 23:31:13 smithi102 podman[133310]: 2024-04-24 23:31:13.527242583 +0000 UTC m=+3.275841454 container cleanup 4ae03cbd359f9d19018af8ec9128126154c77c6628fb42dd3f5ae48ba6650f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-24T23:31:13.769 INFO:journalctl@ceph.mds.j.smithi102.stdout:Apr 24 23:31:13 smithi102 bash[133310]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-j 2024-04-24T23:31:13.897 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.j.service' 2024-04-24T23:31:13.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:13.928 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-24T23:31:13.928 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-24T23:31:13.928 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.b 2024-04-24T23:31:14.225 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:31:13 smithi161 systemd[1]: Stopping Ceph mds.b for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:14.482 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:31:14 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b[63009]: 2024-04-24T23:31:14.224+0000 7f6e91d77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:14.482 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:31:14 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b[63009]: 2024-04-24T23:31:14.224+0000 7f6e91d77700 -1 mds.b *** got signal Terminated *** 2024-04-24T23:31:18.276 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:31:17 smithi161 podman[70093]: 2024-04-24 23:31:17.98528274 +0000 UTC m=+3.857753713 container stop 3182ad60e6179bf9662c5f7a1c8cb13d76055465ed961a625cd1ed8630fbb25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T23:31:18.276 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:31:18 smithi161 podman[70093]: 2024-04-24 23:31:18.011234371 +0000 UTC m=+3.883705346 container died 3182ad60e6179bf9662c5f7a1c8cb13d76055465ed961a625cd1ed8630fbb25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T23:31:18.608 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:31:18 smithi161 podman[70093]: 2024-04-24 23:31:18.374804572 +0000 UTC m=+4.247275534 container cleanup 3182ad60e6179bf9662c5f7a1c8cb13d76055465ed961a625cd1ed8630fbb25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True) 2024-04-24T23:31:18.608 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 24 23:31:18 smithi161 bash[70093]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-b 2024-04-24T23:31:18.798 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.b.service' 2024-04-24T23:31:18.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:18.833 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-24T23:31:18.834 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-24T23:31:18.834 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.e 2024-04-24T23:31:19.145 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:31:18 smithi161 systemd[1]: Stopping Ceph mds.e for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:19.608 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:31:19 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e[62390]: 2024-04-24T23:31:19.144+0000 7f79d702e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:19.608 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:31:19 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e[62390]: 2024-04-24T23:31:19.144+0000 7f79d702e700 -1 mds.e *** got signal Terminated *** 2024-04-24T23:31:22.311 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:31:21 smithi161 podman[70226]: 2024-04-24 23:31:21.992014368 +0000 UTC m=+2.942754152 container died 623b1dcc038311cb77ddbbb32d22d8d829cc3abc23a9c3367f782486d4ccc6f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:31:22.311 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:31:22 smithi161 podman[70226]: 2024-04-24 23:31:22.311330938 +0000 UTC m=+3.262070733 container cleanup 623b1dcc038311cb77ddbbb32d22d8d829cc3abc23a9c3367f782486d4ccc6f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T23:31:22.608 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:31:22 smithi161 bash[70226]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e 2024-04-24T23:31:22.608 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 24 23:31:22 smithi161 podman[70240]: 2024-04-24 23:31:22.512421466 +0000 UTC m=+0.520243556 container remove 623b1dcc038311cb77ddbbb32d22d8d829cc3abc23a9c3367f782486d4ccc6f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:31:22.742 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.e.service' 2024-04-24T23:31:22.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:22.775 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-24T23:31:22.776 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-24T23:31:22.776 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.h 2024-04-24T23:31:23.098 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:31:22 smithi161 systemd[1]: Stopping Ceph mds.h for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:23.358 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:31:23 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h[62698]: 2024-04-24T23:31:23.097+0000 7fb119a4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:23.358 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:31:23 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h[62698]: 2024-04-24T23:31:23.097+0000 7fb119a4a700 -1 mds.h *** got signal Terminated *** 2024-04-24T23:31:26.257 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:31:25 smithi161 podman[70347]: 2024-04-24 23:31:25.991658746 +0000 UTC m=+2.996925040 container died 0edfb28a372e6632af5011b922a033bf600bc23d1cc6052c42f93667bbb940b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T23:31:26.598 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:31:26 smithi161 podman[70347]: 2024-04-24 23:31:26.3566723 +0000 UTC m=+3.361938613 container cleanup 0edfb28a372e6632af5011b922a033bf600bc23d1cc6052c42f93667bbb940b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:31:26.599 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 24 23:31:26 smithi161 bash[70347]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-h 2024-04-24T23:31:26.724 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.h.service' 2024-04-24T23:31:26.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:26.755 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-24T23:31:26.755 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-24T23:31:26.756 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.k 2024-04-24T23:31:26.858 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:31:26 smithi161 systemd[1]: Stopping Ceph mds.k for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:27.358 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:31:27 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-k[62074]: 2024-04-24T23:31:27.067+0000 7f8cbeca3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:27.358 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:31:27 smithi161 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-k[62074]: 2024-04-24T23:31:27.067+0000 7f8cbeca3700 -1 mds.k *** got signal Terminated *** 2024-04-24T23:31:30.602 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:31:30 smithi161 podman[70467]: 2024-04-24 23:31:30.334693826 +0000 UTC m=+3.368383961 container died 15bbc090efde0afe2d53f863cb88884cf22617cf6a599caf240e1cc618c20436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:31:30.858 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:31:30 smithi161 podman[70467]: 2024-04-24 23:31:30.69360224 +0000 UTC m=+3.727292370 container cleanup 15bbc090efde0afe2d53f863cb88884cf22617cf6a599caf240e1cc618c20436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-k, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:31:30.858 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 24 23:31:30 smithi161 bash[70467]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-k 2024-04-24T23:31:31.088 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.k.service' 2024-04-24T23:31:31.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:31.119 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-24T23:31:31.119 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-24T23:31:31.119 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.c 2024-04-24T23:31:31.417 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:31:31 smithi183 systemd[1]: Stopping Ceph mds.c for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:31.761 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:31:31 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c[63377]: 2024-04-24T23:31:31.416+0000 7f71c800e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:31.761 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:31:31 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c[63377]: 2024-04-24T23:31:31.416+0000 7f71c800e700 -1 mds.c *** got signal Terminated *** 2024-04-24T23:31:34.011 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:31:33 smithi183 podman[71650]: 2024-04-24 23:31:33.754119084 +0000 UTC m=+2.442595041 container died f2e55b8c423ee779ede4fdd0c5f3b8713642933f4b58ed3221778f8691b73376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:31:34.659 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:31:34 smithi183 podman[71650]: 2024-04-24 23:31:34.358917472 +0000 UTC m=+3.047393416 container cleanup f2e55b8c423ee779ede4fdd0c5f3b8713642933f4b58ed3221778f8691b73376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-24T23:31:34.659 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 24 23:31:34 smithi183 bash[71650]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-c 2024-04-24T23:31:34.774 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.c.service' 2024-04-24T23:31:34.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:34.809 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-24T23:31:34.809 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-24T23:31:34.810 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.f 2024-04-24T23:31:35.011 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:31:34 smithi183 systemd[1]: Stopping Ceph mds.f for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:35.511 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:31:35 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f[63071]: 2024-04-24T23:31:35.129+0000 7f17b41c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:35.511 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:31:35 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f[63071]: 2024-04-24T23:31:35.129+0000 7f17b41c1700 -1 mds.f *** got signal Terminated *** 2024-04-24T23:31:38.011 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:31:37 smithi183 podman[71783]: 2024-04-24 23:31:37.753215866 +0000 UTC m=+2.726893234 container died 57a123290fd9f2afb88b13559de85e2ef5eaa24d00dac78e4281cd2a79e4f56b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T23:31:38.397 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:31:38 smithi183 podman[71783]: 2024-04-24 23:31:38.121159002 +0000 UTC m=+3.094836364 container cleanup 57a123290fd9f2afb88b13559de85e2ef5eaa24d00dac78e4281cd2a79e4f56b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:31:38.397 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 24 23:31:38 smithi183 bash[71783]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-f 2024-04-24T23:31:38.557 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.f.service' 2024-04-24T23:31:38.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:38.590 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-24T23:31:38.590 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-24T23:31:38.590 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.i 2024-04-24T23:31:38.908 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:31:38 smithi183 systemd[1]: Stopping Ceph mds.i for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:39.261 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:31:38 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i[63686]: 2024-04-24T23:31:38.907+0000 7fc65cde9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:39.261 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:31:38 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i[63686]: 2024-04-24T23:31:38.907+0000 7fc65cde9700 -1 mds.i *** got signal Terminated *** 2024-04-24T23:31:42.011 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:31:41 smithi183 podman[71905]: 2024-04-24 23:31:41.756876451 +0000 UTC m=+2.948751810 container died a1bcedce8d3f3c41a129a04b0da550f37fd40031b07a8c08f7ea89c4645003b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T23:31:42.392 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:31:42 smithi183 podman[71905]: 2024-04-24 23:31:42.116135712 +0000 UTC m=+3.308011055 container cleanup a1bcedce8d3f3c41a129a04b0da550f37fd40031b07a8c08f7ea89c4645003b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-24T23:31:42.392 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 24 23:31:42 smithi183 bash[71905]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-i 2024-04-24T23:31:42.562 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.i.service' 2024-04-24T23:31:42.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:42.594 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-24T23:31:42.594 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-24T23:31:42.594 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.l 2024-04-24T23:31:43.011 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:31:42 smithi183 systemd[1]: Stopping Ceph mds.l for 97eecd66-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:43.511 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:31:43 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-l[62762]: 2024-04-24T23:31:43.011+0000 7f3f33384700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:43.511 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:31:43 smithi183 ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-l[62762]: 2024-04-24T23:31:43.011+0000 7f3f33384700 -1 mds.l *** got signal Terminated *** 2024-04-24T23:31:47.295 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:31:47 smithi183 podman[72026]: 2024-04-24 23:31:47.026306428 +0000 UTC m=+4.221480504 container died a897049076633700cfbd76d084a1fff148997de49be7ccf5c3410b4a37703b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T23:31:47.637 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:31:47 smithi183 podman[72026]: 2024-04-24 23:31:47.394906643 +0000 UTC m=+4.590080721 container cleanup a897049076633700cfbd76d084a1fff148997de49be7ccf5c3410b4a37703b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-04-24T23:31:47.638 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 24 23:31:47 smithi183 bash[72026]: ceph-97eecd66-028e-11ef-bc93-c7b262605968-mds-l 2024-04-24T23:31:47.747 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97eecd66-028e-11ef-bc93-c7b262605968@mds.l.service' 2024-04-24T23:31:47.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:47.777 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-24T23:31:47.778 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97eecd66-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:31:47.963 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:31:50.584 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97eecd66-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:31:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:31:53.145 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97eecd66-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:31:53.337 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:31:55.757 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:31:55.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:31:55.819 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:31:55.845 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T23:31:55.849 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/remote/smithi102/crash 2024-04-24T23:31:55.850 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:31:55.894 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:31:55.894 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:31:55.896 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/remote/smithi161/crash 2024-04-24T23:31:55.897 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:31:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:31:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:31:55.934 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/remote/smithi183/crash 2024-04-24T23:31:55.935 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:31:55.966 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/97eecd66-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:31:55.966 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:31:55.967 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T23:31:55.967 DEBUG:teuthology.orchestra.run.smithi102:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T23:31:56.046 INFO:tasks.cephadm:Compressing logs... 2024-04-24T23:31:56.047 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:31:56.049 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:31:56.052 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:31:56.091 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T23:31:56.091 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T23:31:56.102 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:31:56.102 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T23:31:56.102 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:31:56.103 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr: 87.2% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-24T23:31:56.104 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:31:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T23:31:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:31:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:31:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-24T23:31:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-24T23:31:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T23:31:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T23:31:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-24T23:31:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-24T23:31:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-24T23:31:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:31:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-24T23:31:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:31:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-24T23:31:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-24T23:31:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:31:56.181 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 83.8% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:31:56.202 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 90.2% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:31:56.215 INFO:teuthology.orchestra.run.smithi102.stderr: 84.1% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:31:56.225 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:31:56.319 INFO:teuthology.orchestra.run.smithi102.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:31:56.333 INFO:teuthology.orchestra.run.smithi183.stderr: 91.0% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:31:56.344 INFO:teuthology.orchestra.run.smithi161.stderr: 92.3% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-24T23:31:56.345 INFO:teuthology.orchestra.run.smithi183.stderr: 90.3% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:31:56.345 INFO:teuthology.orchestra.run.smithi161.stderr: 91.1% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:31:56.396 INFO:teuthology.orchestra.run.smithi102.stderr: 91.1% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:31:56.417 INFO:teuthology.orchestra.run.smithi183.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:31:56.427 INFO:teuthology.orchestra.run.smithi161.stderr: 94.5% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-24T23:31:56.432 INFO:teuthology.orchestra.run.smithi161.stderr: 83.8% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:31:56.445 INFO:teuthology.orchestra.run.smithi161.stderr: 87.3% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:31:56.450 INFO:teuthology.orchestra.run.smithi183.stderr: 92.4% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-24T23:31:56.454 INFO:teuthology.orchestra.run.smithi183.stderr: 94.7% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-24T23:31:56.456 INFO:teuthology.orchestra.run.smithi183.stderr: 87.3% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:31:56.480 INFO:teuthology.orchestra.run.smithi161.stderr: 90.3% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:31:56.560 INFO:teuthology.orchestra.run.smithi102.stderr: 94.7% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-24T23:31:56.693 INFO:teuthology.orchestra.run.smithi161.stderr: 94.3% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-24T23:31:56.928 INFO:teuthology.orchestra.run.smithi102.stderr: 94.8% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-24T23:31:56.928 INFO:teuthology.orchestra.run.smithi102.stderr: 94.7% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-24T23:31:56.935 INFO:teuthology.orchestra.run.smithi102.stderr: 94.7% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-24T23:31:58.496 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-24T23:31:58.785 INFO:teuthology.orchestra.run.smithi183.stderr: 93.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-24T23:31:58.983 INFO:teuthology.orchestra.run.smithi161.stderr: 92.0% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-24T23:31:59.492 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi102.stderr: 90.3% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-24T23:32:03.763 INFO:teuthology.orchestra.run.smithi183.stderr: 94.5% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-24T23:32:07.677 INFO:teuthology.orchestra.run.smithi161.stderr: 93.3% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-24T23:32:08.894 INFO:teuthology.orchestra.run.smithi102.stderr: 90.4% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-24T23:33:22.683 INFO:teuthology.orchestra.run.smithi183.stderr: 93.7% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-24T23:33:36.908 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-24T23:34:00.699 INFO:teuthology.orchestra.run.smithi102.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T23:35:02.607 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T23:37:21.376 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-24T23:37:29.496 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T23:37:29.497 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-04-24T23:37:29.498 INFO:teuthology.orchestra.run.smithi183.stderr:real 5m33.430s 2024-04-24T23:37:29.498 INFO:teuthology.orchestra.run.smithi183.stderr:user 6m25.610s 2024-04-24T23:37:29.498 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m14.886s 2024-04-24T23:38:01.252 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-24T23:41:36.545 INFO:teuthology.orchestra.run.smithi161.stderr: 92.0% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T23:42:53.885 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-24T23:43:37.050 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T23:43:37.051 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-04-24T23:43:37.051 INFO:teuthology.orchestra.run.smithi161.stderr:real 11m40.986s 2024-04-24T23:43:37.051 INFO:teuthology.orchestra.run.smithi161.stderr:user 6m34.144s 2024-04-24T23:43:37.052 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m16.430s 2024-04-24T23:45:39.497 INFO:teuthology.orchestra.run.smithi102.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-24T23:46:16.007 INFO:teuthology.orchestra.run.smithi102.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-24T23:47:51.429 INFO:teuthology.orchestra.run.smithi102.stderr: 91.9% -- replaced with /var/log/ceph/97eecd66-028e-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T23:49:57.520 INFO:teuthology.orchestra.run.smithi102.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-24T23:49:58.945 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-04-24T23:49:58.945 INFO:teuthology.orchestra.run.smithi102.stderr:real 18m1.456s 2024-04-24T23:49:58.945 INFO:teuthology.orchestra.run.smithi102.stderr:user 8m50.062s 2024-04-24T23:49:58.946 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m24.610s 2024-04-24T23:49:58.946 INFO:tasks.cephadm:Archiving logs... 2024-04-24T23:49:58.979 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/remote/smithi102/log 2024-04-24T23:49:58.985 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:51:23.018 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/remote/smithi161/log 2024-04-24T23:51:23.024 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:52:31.878 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/remote/smithi183/log 2024-04-24T23:52:31.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:53:36.515 INFO:tasks.cephadm:Removing cluster... 2024-04-24T23:53:36.515 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97eecd66-028e-11ef-bc93-c7b262605968 --force 2024-04-24T23:53:36.912 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:53:37.576 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97eecd66-028e-11ef-bc93-c7b262605968 --force 2024-04-24T23:53:37.873 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:53:38.589 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97eecd66-028e-11ef-bc93-c7b262605968 --force 2024-04-24T23:53:38.868 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 97eecd66-028e-11ef-bc93-c7b262605968 2024-04-24T23:53:39.568 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T23:53:39.569 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:53:39.587 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:53:39.606 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:53:39.621 INFO:tasks.cephadm:Teardown complete 2024-04-24T23:53:39.622 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T23:53:39.636 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T23:53:39.637 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:53:39.639 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:53:39.649 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:53:39.770 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T23:53:39.780 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T23:53:39.780 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:53:39.783 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:53:39.786 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:53:39.797 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:53:39.799 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:53:39.800 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:53:39.817 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:53:39.818 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-04-24T23:53:39.818 INFO:teuthology.orchestra.run.smithi183.stdout:^- hv01.front.sepia.ceph.com 2 6 377 24 +918us[ +918us] +/- 573ms 2024-04-24T23:53:39.818 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 +99us[ +98us] +/- 64ms 2024-04-24T23:53:39.818 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -95us[ -95us] +/- 67ms 2024-04-24T23:53:39.818 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:53:39.846 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:53:39.846 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-24T23:53:39.846 INFO:teuthology.orchestra.run.smithi102.stdout:^- hv01.front.sepia.ceph.com 2 6 377 27 +905us[ +905us] +/- 573ms 2024-04-24T23:53:39.846 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 +85us[ +85us] +/- 65ms 2024-04-24T23:53:39.846 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -97us[ -96us] +/- 67ms 2024-04-24T23:53:39.846 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:53:39.852 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:53:39.852 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-04-24T23:53:39.852 INFO:teuthology.orchestra.run.smithi161.stdout:^- hv01.front.sepia.ceph.com 2 6 377 27 +952us[ +960us] +/- 573ms 2024-04-24T23:53:39.853 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 +103us[ +112us] +/- 64ms 2024-04-24T23:53:39.853 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -87us[ -87us] +/- 67ms 2024-04-24T23:53:39.853 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:53:39.854 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T23:53:39.865 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T23:53:39.865 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T23:53:39.876 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:53:40.020 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:53:40.109 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:53:40.208 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:53:40.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:53:40.238 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:53:40.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:53:40.268 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:53:40.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:53:40.294 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T23:53:40.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T23:53:40.320 INFO:teuthology.task.internal:Duration was 3486.917622 seconds 2024-04-24T23:53:40.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T23:53:40.331 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T23:53:40.332 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:53:40.334 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:53:40.337 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:53:40.378 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:53:40.379 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:53:40.401 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:53:40.667 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T23:53:40.667 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-04-24T23:53:40.668 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:53:40.697 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-04-24T23:53:40.697 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:53:40.723 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-04-24T23:53:40.723 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:53:40.750 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T23:53:40.750 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:53:40.753 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:53:40.765 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:53:40.816 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T23:53:40.816 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:53:40.821 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:53:40.833 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:53:41.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T23:53:41.195 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T23:53:41.195 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:53:41.220 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:53:41.253 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:53:41.277 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T23:53:41.289 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:53:41.292 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:53:41.296 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:53:41.326 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-04-24T23:53:41.349 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-04-24T23:53:41.364 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-04-24T23:53:41.384 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:53:41.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:53:41.398 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:53:41.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:53:41.412 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:53:41.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:53:41.426 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T23:53:41.438 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T23:53:41.440 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/remote/smithi102 2024-04-24T23:53:41.440 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:53:41.482 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/remote/smithi161 2024-04-24T23:53:41.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:53:41.519 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672178/remote/smithi183 2024-04-24T23:53:41.519 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:53:41.555 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T23:53:41.556 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:53:41.558 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:53:41.560 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:53:41.603 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T23:53:41.617 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T23:53:41.618 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T23:53:41.636 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T23:53:41.637 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:53:41.639 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:53:41.641 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:53:41.651 INFO:teuthology.orchestra.run.smithi102.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:53 /home/ubuntu/cephtest 2024-04-24T23:53:41.653 INFO:teuthology.orchestra.run.smithi161.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:53 /home/ubuntu/cephtest 2024-04-24T23:53:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:53 /home/ubuntu/cephtest 2024-04-24T23:53:41.658 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T23:53:41.670 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T23:53:41.696 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} duration: 3486.9176223278046 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-24T23:53:41.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:53:41.779 INFO:teuthology.run:pass